[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:14 dist-test-slave-chsk krb5kdc[21621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:14 dist-test-slave-chsk krb5kdc[21621](info): set up 2 sockets
Aug 09 22:43:14 dist-test-slave-chsk krb5kdc[21621](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 22:43:16.014339 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.005s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:16 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:16.207226 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41665
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:41665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:16.531599 21634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:16.531929 21634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:16.532089 21634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:16.539553 21634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:16.539723 21634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:16.539812 21634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:16.539911 21634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:16.539992 21634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:16.549687 21634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:41665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:16.551368 21634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:16.553020 21634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:16.566905 21640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:16.566898 21641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:16.567082 21634 server_base.cc:1034] running on GCE node
W20260809 22:43:16.566834 21643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:16.568125 21634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:16.569474 21634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:16.570708 21634 hybrid_clock.cc:648] HybridClock initialized: now 1786315396570664 us; error 88 us; skew 500 ppm
I20260809 22:43:16.571072 21634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:16 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315396, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:16.574304 21634 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:16.575745 21634 webserver.cc:460] Webserver started at http://127.21.27.190:42591/ using document root <none> and password file <none>
I20260809 22:43:16.576424 21634 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:16.576568 21634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:16.576918 21634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:16.580029 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "8d5138222351448481487be4a6ac8d47"
format_stamp: "Formatted at 2026-08-09 22:43:16 on dist-test-slave-chsk"
server_key: "0196c8aa169db23e9cda17a4121c3f90"
server_key_iv: "3b130455cbff9d33d2ebef4dc0e20048"
server_key_version: "encryptionkey@0"
I20260809 22:43:16.580819 21634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "8d5138222351448481487be4a6ac8d47"
format_stamp: "Formatted at 2026-08-09 22:43:16 on dist-test-slave-chsk"
server_key: "0196c8aa169db23e9cda17a4121c3f90"
server_key_iv: "3b130455cbff9d33d2ebef4dc0e20048"
server_key_version: "encryptionkey@0"
I20260809 22:43:16.585920 21634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:16.589135 21653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:16.590391 21634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:43:16.590585 21634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "8d5138222351448481487be4a6ac8d47"
format_stamp: "Formatted at 2026-08-09 22:43:16 on dist-test-slave-chsk"
server_key: "0196c8aa169db23e9cda17a4121c3f90"
server_key_iv: "3b130455cbff9d33d2ebef4dc0e20048"
server_key_version: "encryptionkey@0"
I20260809 22:43:16.590778 21634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:16.610445 21634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:16.614782 21634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:16.615100 21634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:16.634694 21634 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:41665
I20260809 22:43:16.634718 21707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:41665 every 8 connection(s)
I20260809 22:43:16.636134 21634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:16.639917 21708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:16.645147 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 21634
I20260809 22:43:16.645632 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:16.647527 21614 external_mini_cluster.cc:1428] Setting key 2bbce2803cb79814b6f03d8e383615ba
I20260809 22:43:16.650162 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47: Bootstrap starting.
I20260809 22:43:16.654121 21708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:16.655099 21708 log.cc:824] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:16.657835 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47: No bootstrap required, opened a new log
I20260809 22:43:16.666329 21708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } }
I20260809 22:43:16.666710 21708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:16.666841 21708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d5138222351448481487be4a6ac8d47, State: Initialized, Role: FOLLOWER
I20260809 22:43:16.667465 21708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } }
I20260809 22:43:16.667706 21708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:16.667879 21708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:16.668074 21708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:16.670245 21708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } }
I20260809 22:43:16.670737 21708 leader_election.cc:302] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d5138222351448481487be4a6ac8d47; no voters:
I20260809 22:43:16.671411 21708 leader_election.cc:288] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:16.671538 21713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:16.672843 21713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [term 1 LEADER]: Becoming Leader. State: Replica: 8d5138222351448481487be4a6ac8d47, State: Running, Role: LEADER
I20260809 22:43:16.673455 21713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } }
I20260809 22:43:16.673772 21708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 22:43:16 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:16.676467 21715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d5138222351448481487be4a6ac8d47. Latest consensus state: current_term: 1 leader_uuid: "8d5138222351448481487be4a6ac8d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } } }
I20260809 22:43:16.676916 21715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:16.677600 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d5138222351448481487be4a6ac8d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d5138222351448481487be4a6ac8d47" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41665 } } }
I20260809 22:43:16.677955 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:16.680190 21718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:16.686461 21718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:16.690953 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:16.651357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.21.27.190:41665)
0809 22:43:16.652233 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:16.652251 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:16.653405 (+ 1154us) server_negotiation.cc:407] Connection header received
0809 22:43:16.662497 (+ 9092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:16.662516 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:16.662885 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:16.663290 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:16.666078 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:16.667687 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:16.669422 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:16.670085 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:16.670290 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:16.677623 (+ 7333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:16.677639 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:16.677664 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:16.677728 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:16.682455 (+ 4727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:16.683649 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:16.683659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:16.683667 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:16.684045 (+ 378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:16.684656 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:16.684661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:16.684665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:16.685552 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:16.688696 (+ 3144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:16.688968 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 22:43:16.689408 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":432,"threads_started":1}
I20260809 22:43:16.697831 21718 catalog_manager.cc:1347] Generated new cluster ID: 11bb7e5b94e9485091cf91f73bf647fa
I20260809 22:43:16.697942 21718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:16.718205 21718 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:16.719513 21718 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:16.731576 21718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d5138222351448481487be4a6ac8d47: Generated new TSK 0
I20260809 22:43:16.732496 21718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:16.784762 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:17.120481 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:17.120767 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:17.120949 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:17.128394 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:17.128528 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:17.128700 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:17.138667 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:17.139978 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:17.141335 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:17.157122 21743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:17.157039 21746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:17.157084 21744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:17.159379 21738 server_base.cc:1034] running on GCE node
I20260809 22:43:17.160068 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:17.161072 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:17.162297 21738 hybrid_clock.cc:648] HybridClock initialized: now 1786315397162243 us; error 46 us; skew 500 ppm
I20260809 22:43:17.162588 21738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:17 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:17.165875 21738 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:17.167403 21738 webserver.cc:460] Webserver started at http://127.21.27.129:35451/ using document root <none> and password file <none>
I20260809 22:43:17.168111 21738 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:17.168251 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:17.168648 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:17.171662 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "953d6045cb9d410cb1c03c3d23512ae0"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "c112cf355b9b46bb57d42e2acf25eec6"
server_key_iv: "5ef9e3ce3e6114b429e2777a943e1ed4"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.172457 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "953d6045cb9d410cb1c03c3d23512ae0"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "c112cf355b9b46bb57d42e2acf25eec6"
server_key_iv: "5ef9e3ce3e6114b429e2777a943e1ed4"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.177953 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:43:17.181576 21753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.182940 21738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:43:17.183120 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "953d6045cb9d410cb1c03c3d23512ae0"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "c112cf355b9b46bb57d42e2acf25eec6"
server_key_iv: "5ef9e3ce3e6114b429e2777a943e1ed4"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.183384 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:17.209417 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:17.213642 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:17.213950 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:17.214977 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:17.216452 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:17.216570 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.216713 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:17.216784 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.252035 21738 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:43947
I20260809 22:43:17.252130 21874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:43947 every 8 connection(s)
I20260809 22:43:17.253420 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:17.253808 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 21738
I20260809 22:43:17.254045 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:17.254956 21614 external_mini_cluster.cc:1428] Setting key eb38e51f71b16c917dfe0400e50fc4ec
Aug 09 22:43:17 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:17.273847 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:17.256343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:47545 (local address 127.21.27.190:41665)
0809 22:43:17.256801 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:17.256807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:17.258005 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 22:43:17.260051 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:17.260057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:17.260150 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:17.260262 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:17.261783 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.262498 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.264098 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.264506 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.264643 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:17.268201 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:17.268206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:17.268209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:17.268241 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:17.270848 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:17.271672 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:17.271677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:17.271679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:17.271798 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:17.272234 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:17.272239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:17.272247 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:17.272560 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:17.272715 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:17.273400 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 22:43:17.273589 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":239,"threads_started":1}
I20260809 22:43:17.274950 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:17.257010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41665 (local address 127.21.27.129:47545)
0809 22:43:17.257788 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:17.257859 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:17.259749 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:17.260497 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:17.260505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:17.261054 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:17.261615 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:17.261631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.262661 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.262666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:17.263942 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:17.263953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.264744 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.264751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:17.264970 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:17.265693 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:17.265717 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:17.267951 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:17.271004 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:17.271013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:17.271028 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:17.271540 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:17.271911 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:17.271916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:17.271920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:17.272115 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:17.272695 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:17.272702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:17.273019 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 22:43:17.273314 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 22:43:17.273738 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":334,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
I20260809 22:43:17.276487 21875 heartbeater.cc:347] Connected to a master server at 127.21.27.190:41665
I20260809 22:43:17.276813 21875 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:17.277485 21875 heartbeater.cc:511] Master 127.21.27.190:41665 requested a full tablet report, sending...
I20260809 22:43:17.279527 21671 ts_manager.cc:194] Registered new tserver with Master: 953d6045cb9d410cb1c03c3d23512ae0 (127.21.27.129:43947)
I20260809 22:43:17.280900 21671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:47545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:17.298808 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:17.618531 21881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:17.618813 21881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:17.618966 21881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:17.626341 21881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:17.626467 21881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:17.626673 21881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:17.636493 21881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:17.637655 21881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:17.638914 21881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:17.651731 21886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:17.651714 21887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:17.651693 21889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:17.652805 21881 server_base.cc:1034] running on GCE node
I20260809 22:43:17.653398 21881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:17.654357 21881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:17.655566 21881 hybrid_clock.cc:648] HybridClock initialized: now 1786315397655516 us; error 37 us; skew 500 ppm
I20260809 22:43:17.655891 21881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:17 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:17.659083 21881 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:17.660566 21881 webserver.cc:460] Webserver started at http://127.21.27.130:40091/ using document root <none> and password file <none>
I20260809 22:43:17.661259 21881 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:17.661402 21881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:17.661715 21881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:17.664781 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "55bddc7720a54c4eb047a8a3bd035c1a"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "759f734c51e8dab39e60fd7d2429c811"
server_key_iv: "e2afec8194ea9697ee0c5af3508086c8"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.665532 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "55bddc7720a54c4eb047a8a3bd035c1a"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "759f734c51e8dab39e60fd7d2429c811"
server_key_iv: "e2afec8194ea9697ee0c5af3508086c8"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.670581 21881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:17.673947 21896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.675405 21881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:17.675572 21881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "55bddc7720a54c4eb047a8a3bd035c1a"
format_stamp: "Formatted at 2026-08-09 22:43:17 on dist-test-slave-chsk"
server_key: "759f734c51e8dab39e60fd7d2429c811"
server_key_iv: "e2afec8194ea9697ee0c5af3508086c8"
server_key_version: "encryptionkey@0"
I20260809 22:43:17.675762 21881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:17.699393 21881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:17.704130 21881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:17.704432 21881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:17.705493 21881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:17.706835 21881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:17.706938 21881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.707093 21881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:17.707181 21881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:17.744894 21881 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:39287
I20260809 22:43:17.744983 22016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:39287 every 8 connection(s)
I20260809 22:43:17.746244 21881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:17.756263 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 21881
I20260809 22:43:17.756487 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:17.757071 21614 external_mini_cluster.cc:1428] Setting key 5fb559667bc2f099b44ad7570e03e23b
Aug 09 22:43:17 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:17.766800 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:17.748737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:50867 (local address 127.21.27.190:41665)
0809 22:43:17.748904 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:17.748909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:17.750031 (+ 1122us) server_negotiation.cc:407] Connection header received
0809 22:43:17.751950 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:17.751954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:17.752028 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:17.752126 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:17.753549 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.754247 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.755809 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.756257 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.756384 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:17.760645 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:17.760650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:17.760653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:17.760685 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:17.763483 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:17.764415 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:17.764419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:17.764421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:17.764546 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:17.765106 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:17.765113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:17.765119 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:17.765524 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:17.765687 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:17.766347 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 22:43:17.766519 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:43:17.767823 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:17.749129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41665 (local address 127.21.27.130:50867)
0809 22:43:17.749852 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:17.749898 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:17.751688 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:17.752320 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:17.752328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:17.752861 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:17.753394 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:17.753411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.754375 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.754379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:17.755679 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:17.755691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:17.756503 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:17.756513 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:17.756797 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:17.757534 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:17.757564 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:17.760414 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:17.763622 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:17.763634 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:17.763653 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:17.764297 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:17.764654 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:17.764661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:17.764667 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:17.764930 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:17.765668 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:17.765674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:17.765976 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 22:43:17.766214 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 22:43:17.766614 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":268,"threads_started":1}
I20260809 22:43:17.769304 22017 heartbeater.cc:347] Connected to a master server at 127.21.27.190:41665
I20260809 22:43:17.769635 22017 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:17.770519 22017 heartbeater.cc:511] Master 127.21.27.190:41665 requested a full tablet report, sending...
I20260809 22:43:17.772200 21671 ts_manager.cc:194] Registered new tserver with Master: 55bddc7720a54c4eb047a8a3bd035c1a (127.21.27.130:39287)
I20260809 22:43:17.773525 21671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:50867
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:17.803977 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:18.128410 22022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:18.128680 22022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:18.128836 22022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:18.136101 22022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:18.136211 22022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:18.136375 22022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:18.146212 22022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:41665
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:18.147650 22022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:18.149015 22022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:18.162002 22028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:18.161898 22032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:18.162081 22030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:18.162760 22022 server_base.cc:1034] running on GCE node
I20260809 22:43:18.163417 22022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:18.164358 22022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:18.165581 22022 hybrid_clock.cc:648] HybridClock initialized: now 1786315398165520 us; error 54 us; skew 500 ppm
I20260809 22:43:18.165910 22022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:18.169082 22022 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:18.170368 22022 webserver.cc:460] Webserver started at http://127.21.27.131:35259/ using document root <none> and password file <none>
I20260809 22:43:18.171003 22022 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:18.171105 22022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:18.171427 22022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:18.174367 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "598451934a0f42c48c91f312982a160c"
format_stamp: "Formatted at 2026-08-09 22:43:18 on dist-test-slave-chsk"
server_key: "3ff90174a7453da2960f10cc2564f888"
server_key_iv: "36aa70a4055c3e4f9e2fef4c82349f4c"
server_key_version: "encryptionkey@0"
I20260809 22:43:18.175110 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "598451934a0f42c48c91f312982a160c"
format_stamp: "Formatted at 2026-08-09 22:43:18 on dist-test-slave-chsk"
server_key: "3ff90174a7453da2960f10cc2564f888"
server_key_iv: "36aa70a4055c3e4f9e2fef4c82349f4c"
server_key_version: "encryptionkey@0"
I20260809 22:43:18.180184 22022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:43:18.183465 22039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:18.184635 22022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 22:43:18.184755 22022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "598451934a0f42c48c91f312982a160c"
format_stamp: "Formatted at 2026-08-09 22:43:18 on dist-test-slave-chsk"
server_key: "3ff90174a7453da2960f10cc2564f888"
server_key_iv: "36aa70a4055c3e4f9e2fef4c82349f4c"
server_key_version: "encryptionkey@0"
I20260809 22:43:18.184900 22022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:18.207122 22022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:18.210829 22022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:18.211071 22022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:18.212033 22022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:18.213253 22022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:18.213325 22022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:18.213404 22022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:18.213451 22022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:18.248932 22022 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45229
I20260809 22:43:18.249024 22157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45229 every 8 connection(s)
I20260809 22:43:18.250250 22022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:18.252310 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22022
I20260809 22:43:18.252526 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:18.253127 21614 external_mini_cluster.cc:1428] Setting key 15d32b5e8d6f1788bc253ae60f4ed2a2
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:18.272709 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.253560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:46491 (local address 127.21.27.190:41665)
0809 22:43:18.254139 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.254146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.255109 (+ 963us) server_negotiation.cc:407] Connection header received
0809 22:43:18.257640 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.257644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.257709 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.257792 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.259243 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.259992 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.262124 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.262535 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.262674 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.266200 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.266205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.266208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.266241 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.269586 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.270326 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.270331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.270335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.270469 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.270901 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.270906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.270909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.271257 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.271424 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.272129 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.272357 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408}
I20260809 22:43:18.273806 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.254106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41665 (local address 127.21.27.131:46491)
0809 22:43:18.254953 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:18.254994 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:18.257381 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:18.257987 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:18.257995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:18.258521 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:18.259062 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:18.259078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.260211 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.260217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:18.261946 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:18.261964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.262764 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.262774 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:18.263010 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.263769 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:18.263791 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:18.266003 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:18.269727 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:18.269737 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:18.269754 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:18.270196 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:18.270581 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:18.270586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:18.270589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:18.270781 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:18.271404 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:18.271411 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:18.271759 (+ 348us) client_negotiation.cc:769] Sending connection context
0809 22:43:18.272027 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 22:43:18.272566 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":360,"threads_started":1}
I20260809 22:43:18.275208 22158 heartbeater.cc:347] Connected to a master server at 127.21.27.190:41665
I20260809 22:43:18.275557 22158 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:18.276281 22158 heartbeater.cc:511] Master 127.21.27.190:41665 requested a full tablet report, sending...
I20260809 22:43:18.277678 21671 ts_manager.cc:194] Registered new tserver with Master: 598451934a0f42c48c91f312982a160c (127.21.27.131:45229)
I20260809 22:43:18.278473 21671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:46491
I20260809 22:43:18.281905 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:43:18.284525 21875 heartbeater.cc:503] Master 127.21.27.190:41665 was elected leader, sending a full tablet report...
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:18.331044 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.314904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.21.27.129:43947)
0809 22:43:18.315622 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.315629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.315657 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:43:18.315757 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.315763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.315964 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.316209 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.317806 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.318811 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.320228 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.320778 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.320905 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.324903 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.324916 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.324930 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.324967 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.328209 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.328958 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.328965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.328968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.329110 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.329475 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.329478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.329480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.329825 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.330054 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.330482 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.330688 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":387,"threads_started":1}
W20260809 22:43:18.332407 21784 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56974
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:18.381414 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.367003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56984 (local address 127.21.27.129:43947)
0809 22:43:18.367301 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.367310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.367550 (+ 240us) server_negotiation.cc:407] Connection header received
0809 22:43:18.367778 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.367782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.367878 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.367988 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.369415 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.370119 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.371440 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.371871 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.372008 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.376027 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.376031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.376035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.376075 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.378604 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.379322 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.379327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.379329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.379513 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.379899 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.379904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.379908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.380233 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.380480 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.380839 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.381084 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:43:18.383261 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 21738
I20260809 22:43:18.396250 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.384296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.21.27.129:43947)
0809 22:43:18.384520 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.384527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.384762 (+ 235us) server_negotiation.cc:407] Connection header received
0809 22:43:18.384939 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.384943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.384998 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.385102 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.386272 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.387047 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.388363 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.388770 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.388900 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.391135 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.391178 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.391185 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.391224 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.393627 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.394280 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.394287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.394290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.394440 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.394875 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.394879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.394881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.395201 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.395395 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.395643 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.395816 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:43:18.396997 21870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:18.482347 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 21738
I20260809 22:43:18.501503 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 21881
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:18.517170 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.502478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.21.27.130:39287)
0809 22:43:18.502963 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.502970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.502994 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:43:18.503053 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.503059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.503319 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.503485 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.504675 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.505695 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.506961 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.507638 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.507773 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.512039 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.512049 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.512062 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.512111 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.514672 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.515284 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.515290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.515292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.515429 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.515771 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.515774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.515775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.516091 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.516313 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.516553 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.516741 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":254,"threads_started":1}
I20260809 22:43:18.518499 22011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:18.599685 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 21881
I20260809 22:43:18.617559 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22022
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[21621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:18.635058 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:18.618703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.21.27.131:45229)
0809 22:43:18.619409 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:18.619416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:18.619446 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:43:18.619504 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:18.619510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:18.619718 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:18.619842 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:18.621262 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.622324 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.623624 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:18.624194 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:18.624289 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:18.628406 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:18.628418 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:18.628433 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:18.628472 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:18.631850 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.632677 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.632687 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.632691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.632856 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:18.633235 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:18.633243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:18.633248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:18.633725 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:18.634135 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:18.634394 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 22:43:18.634653 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":348,"threads_started":1}
I20260809 22:43:18.636742 22153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:18.720404 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22022
I20260809 22:43:18.738869 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 21634
I20260809 22:43:18.740164 21703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:18.844128 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 21634
2026-08-09T22:43:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4009 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[22181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[22181](info): set up 2 sockets
Aug 09 22:43:18 dist-test-slave-chsk krb5kdc[22181](info): commencing operation
krb5kdc: starting...
W20260809 22:43:20.000705 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:20 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:20.186618 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40805
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:20.505941 22195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:20.506313 22195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:20.506464 22195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:20.513922 22195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:20.514072 22195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:20.514169 22195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:20.514262 22195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:20.514350 22195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:20.524457 22195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:20.525784 22195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:20.527071 22195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:20.539206 22204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:20.539170 22201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:20.539122 22202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:20.540426 22195 server_base.cc:1034] running on GCE node
I20260809 22:43:20.541165 22195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:20.542557 22195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:20.543766 22195 hybrid_clock.cc:648] HybridClock initialized: now 1786315400543715 us; error 33 us; skew 500 ppm
I20260809 22:43:20.544088 22195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:20 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:20.551261 22195 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:20.552837 22195 webserver.cc:460] Webserver started at http://127.21.27.190:37809/ using document root <none> and password file <none>
I20260809 22:43:20.553539 22195 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:20.553658 22195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:20.553931 22195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:20.557444 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0a7d3f767f334ab78fdfb9895658d776"
format_stamp: "Formatted at 2026-08-09 22:43:20 on dist-test-slave-chsk"
server_key: "7ac259b2944506bb74311a5d01563dff"
server_key_iv: "adcd83a3ef871c99a32abcadde911f94"
server_key_version: "encryptionkey@0"
I20260809 22:43:20.558168 22195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0a7d3f767f334ab78fdfb9895658d776"
format_stamp: "Formatted at 2026-08-09 22:43:20 on dist-test-slave-chsk"
server_key: "7ac259b2944506bb74311a5d01563dff"
server_key_iv: "adcd83a3ef871c99a32abcadde911f94"
server_key_version: "encryptionkey@0"
I20260809 22:43:20.563354 22195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 22:43:20.566802 22213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:20.568182 22195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:20.568377 22195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0a7d3f767f334ab78fdfb9895658d776"
format_stamp: "Formatted at 2026-08-09 22:43:20 on dist-test-slave-chsk"
server_key: "7ac259b2944506bb74311a5d01563dff"
server_key_iv: "adcd83a3ef871c99a32abcadde911f94"
server_key_version: "encryptionkey@0"
I20260809 22:43:20.568588 22195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:20.586771 22195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:20.590981 22195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:20.591339 22195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:20.610594 22195 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40805
I20260809 22:43:20.610601 22269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40805 every 8 connection(s)
I20260809 22:43:20.612075 22195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:20.614349 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22195
I20260809 22:43:20.614601 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:20.615340 21614 external_mini_cluster.cc:1428] Setting key 50e87398be6f2c915e1b30772b7c17d5
I20260809 22:43:20.616182 22270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:20.631001 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776: Bootstrap starting.
Aug 09 22:43:20 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:20.634238 22270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:20.635396 22270 log.cc:824] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:20.638339 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776: No bootstrap required, opened a new log
I20260809 22:43:20.639673 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:20.617396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.21.27.190:40805)
0809 22:43:20.618226 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:20.618237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:20.618296 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:43:20.620715 (+ 2419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:20.620737 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:20.621150 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:20.621495 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:20.623481 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:20.624774 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:20.626531 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:20.627364 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:20.627502 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:20.632241 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:20.632249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:20.632263 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:20.632301 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:20.635488 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:20.636189 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:20.636202 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:20.636208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:20.636379 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:20.636777 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:20.636781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:20.636784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:20.637320 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:20.637631 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:20.638171 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 22:43:20.638629 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":369,"threads_started":1}
I20260809 22:43:20.646461 22270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } }
I20260809 22:43:20.646879 22270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:20.647032 22270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7d3f767f334ab78fdfb9895658d776, State: Initialized, Role: FOLLOWER
I20260809 22:43:20.647611 22270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } }
I20260809 22:43:20.647857 22270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:20.648000 22270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:20.648175 22270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:20.650388 22270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } }
I20260809 22:43:20.650898 22270 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7d3f767f334ab78fdfb9895658d776; no voters:
I20260809 22:43:20.651549 22270 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:20.651682 22275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:20.652410 22275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [term 1 LEADER]: Becoming Leader. State: Replica: 0a7d3f767f334ab78fdfb9895658d776, State: Running, Role: LEADER
I20260809 22:43:20.652912 22275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } }
I20260809 22:43:20.653743 22270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:20.656028 22277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7d3f767f334ab78fdfb9895658d776. Latest consensus state: current_term: 1 leader_uuid: "0a7d3f767f334ab78fdfb9895658d776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } } }
I20260809 22:43:20.656479 22277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:20.656708 22276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a7d3f767f334ab78fdfb9895658d776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7d3f767f334ab78fdfb9895658d776" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40805 } } }
I20260809 22:43:20.657038 22276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:20.658751 22284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:20.664173 22284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:20.673336 22284 catalog_manager.cc:1347] Generated new cluster ID: 21262ed73c844cda96fdfcf34f35b84b
I20260809 22:43:20.673481 22284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:20.687633 22284 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:20.688961 22284 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:20.699374 22284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7d3f767f334ab78fdfb9895658d776: Generated new TSK 0
I20260809 22:43:20.700446 22284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:20.754978 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:21.079694 22296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:21.079968 22296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:21.080147 22296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:21.087330 22296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:21.087435 22296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:21.087626 22296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:21.097272 22296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:21.098487 22296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:21.099809 22296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:21.113082 22302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:21.113125 22301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:21.113651 22304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:21.116603 22296 server_base.cc:1034] running on GCE node
I20260809 22:43:21.117283 22296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:21.118228 22296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:21.119465 22296 hybrid_clock.cc:648] HybridClock initialized: now 1786315401119415 us; error 61 us; skew 500 ppm
I20260809 22:43:21.119792 22296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:21 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315401, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:21.123040 22296 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:21.124593 22296 webserver.cc:460] Webserver started at http://127.21.27.129:33765/ using document root <none> and password file <none>
I20260809 22:43:21.125319 22296 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:21.125463 22296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:21.125797 22296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:21.129539 22296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "820604f0ad75413cbe23cb4a53536c4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "5d8722131f1a2a446bd877c751f4040f"
server_key_iv: "8c31e2b634e6eaf23684183c2feffab7"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.130486 22296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "820604f0ad75413cbe23cb4a53536c4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "5d8722131f1a2a446bd877c751f4040f"
server_key_iv: "8c31e2b634e6eaf23684183c2feffab7"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.135624 22296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:43:21.139235 22311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.140547 22296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:21.140717 22296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "820604f0ad75413cbe23cb4a53536c4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "5d8722131f1a2a446bd877c751f4040f"
server_key_iv: "8c31e2b634e6eaf23684183c2feffab7"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.140909 22296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:21.162014 22296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:21.165966 22296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:21.166263 22296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:21.167250 22296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:21.168716 22296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:21.168815 22296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.168965 22296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:21.169034 22296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.202764 22296 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:39043
I20260809 22:43:21.202821 22435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:39043 every 8 connection(s)
I20260809 22:43:21.204116 22296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:21.205475 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22296
I20260809 22:43:21.205711 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:21.206593 21614 external_mini_cluster.cc:1428] Setting key 77ad08393530006e41f25ded7bde2e25
Aug 09 22:43:21 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315401, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:21.226631 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:21.206967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50557 (local address 127.21.27.190:40805)
0809 22:43:21.207536 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:21.207543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:21.208674 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 22:43:21.210438 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:21.210446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:21.210534 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:21.210648 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:21.212092 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.213214 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.214732 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.215191 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.215329 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:21.219466 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:21.219475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:21.219483 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:21.219527 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:21.223335 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:21.224084 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:21.224092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:21.224098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:21.224262 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:21.224687 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:21.224693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:21.224697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:21.225064 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:21.225221 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:21.225905 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 22:43:21.226172 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":299,"threads_started":1}
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:21.227299 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:21.207587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40805 (local address 127.21.27.129:50557)
0809 22:43:21.208427 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:21.208486 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:21.210184 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:21.210806 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:21.210813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:21.211382 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:21.211920 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:21.211936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.213382 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.213385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:21.214588 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:21.214600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.215426 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.215434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:21.215770 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:21.216487 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:21.216543 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:21.219224 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:21.223485 (+ 4261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:21.223491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:21.223503 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:21.223943 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:21.224373 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:21.224376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:21.224378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:21.224566 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:21.225183 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:21.225188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:21.225496 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 22:43:21.225721 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 22:43:21.226110 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":379,"threads_started":1}
I20260809 22:43:21.228801 22436 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40805
I20260809 22:43:21.229099 22436 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:21.229816 22436 heartbeater.cc:511] Master 127.21.27.190:40805 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
I20260809 22:43:21.231998 22232 ts_manager.cc:194] Registered new tserver with Master: 820604f0ad75413cbe23cb4a53536c4b (127.21.27.129:39043)
I20260809 22:43:21.233424 22232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:50557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:21.254410 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:21.575062 22442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:21.575416 22442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:21.575570 22442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:21.582681 22442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:21.582824 22442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:21.583006 22442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:21.593115 22442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:21.594410 22442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:21.595754 22442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:21.608536 22450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:21.608562 22448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:21.608628 22447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:21.609493 22442 server_base.cc:1034] running on GCE node
I20260809 22:43:21.610291 22442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:21.611488 22442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:21.612733 22442 hybrid_clock.cc:648] HybridClock initialized: now 1786315401612674 us; error 53 us; skew 500 ppm
I20260809 22:43:21.613106 22442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:21 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315401, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:21.616894 22442 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:21.626804 22442 webserver.cc:460] Webserver started at http://127.21.27.130:40191/ using document root <none> and password file <none>
I20260809 22:43:21.627592 22442 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:21.627749 22442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:21.628067 22442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:21.630987 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "04b1f67b44da4b4ba19017e005ea0b4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "cc7a88816caa3e2c8ece45418d64ec14"
server_key_iv: "f71fb7626d77b631dba22b196bae4e34"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.631856 22442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "04b1f67b44da4b4ba19017e005ea0b4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "cc7a88816caa3e2c8ece45418d64ec14"
server_key_iv: "f71fb7626d77b631dba22b196bae4e34"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.637099 22442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:21.640501 22457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.641790 22442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:21.641960 22442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "04b1f67b44da4b4ba19017e005ea0b4b"
format_stamp: "Formatted at 2026-08-09 22:43:21 on dist-test-slave-chsk"
server_key: "cc7a88816caa3e2c8ece45418d64ec14"
server_key_iv: "f71fb7626d77b631dba22b196bae4e34"
server_key_version: "encryptionkey@0"
I20260809 22:43:21.642174 22442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:21.658215 22442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:21.662563 22442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:21.662858 22442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:21.663857 22442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:21.665131 22442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:21.665237 22442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.665393 22442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:21.665462 22442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:21.698021 22442 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:40837
I20260809 22:43:21.698087 22578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:40837 every 8 connection(s)
I20260809 22:43:21.699368 22442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:21.703784 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22442
I20260809 22:43:21.703980 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:21.704469 21614 external_mini_cluster.cc:1428] Setting key e650a2ab46801406a4e46f6ba74ec63e
Aug 09 22:43:21 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315401, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:21.720927 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:21.701831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43249 (local address 127.21.27.190:40805)
0809 22:43:21.702026 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:21.702031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:21.703101 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 22:43:21.705049 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:21.705055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:21.705170 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:21.705277 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:21.707103 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.708331 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.709810 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.710230 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.710365 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:21.714013 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:21.714022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:21.714030 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:21.714084 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:21.717667 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:21.718381 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:21.718390 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:21.718396 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:21.718584 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:21.718980 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:21.718987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:21.718994 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:21.719425 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:21.719586 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:21.720232 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 22:43:21.720495 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:43:21.721633 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:21.702273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40805 (local address 127.21.27.130:43249)
0809 22:43:21.702945 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:21.702980 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:21.704802 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:21.705549 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:21.705573 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:21.706289 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:21.706931 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:21.706948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.708476 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.708480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:21.709670 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:21.709681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:21.710453 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:21.710460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:21.710687 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:21.711394 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:21.711417 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:21.713775 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:21.717810 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:21.717816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:21.717828 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:21.718255 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:21.718694 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:21.718697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:21.718700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:21.718874 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:21.719545 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:21.719550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:21.719850 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 22:43:21.720074 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 22:43:21.720549 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":267,"threads_started":1}
I20260809 22:43:21.723093 22579 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40805
I20260809 22:43:21.723431 22579 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:21.724125 22579 heartbeater.cc:511] Master 127.21.27.190:40805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
I20260809 22:43:21.725575 22232 ts_manager.cc:194] Registered new tserver with Master: 04b1f67b44da4b4ba19017e005ea0b4b (127.21.27.130:40837)
I20260809 22:43:21.726431 22232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:43249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:21.753809 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:22.085927 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:22.086308 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:22.086484 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:22.093875 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:22.094056 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:22.094250 22584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:22.104183 22584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:22.105566 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:22.106925 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:22.120463 22589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:22.120427 22590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:22.120419 22593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:22.121397 22584 server_base.cc:1034] running on GCE node
I20260809 22:43:22.122093 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:22.123060 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:22.124307 22584 hybrid_clock.cc:648] HybridClock initialized: now 1786315402124256 us; error 44 us; skew 500 ppm
I20260809 22:43:22.124627 22584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:22.127758 22584 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:22.129240 22584 webserver.cc:460] Webserver started at http://127.21.27.131:34169/ using document root <none> and password file <none>
I20260809 22:43:22.129914 22584 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:22.130070 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:22.130398 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:22.133378 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "1574e54da59e432d9014bac7e2d14dc9"
format_stamp: "Formatted at 2026-08-09 22:43:22 on dist-test-slave-chsk"
server_key: "f325a1d55a1b20a70028c6bc0eecf366"
server_key_iv: "e535df36cb4912778409ee0673e512c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:22.134398 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "1574e54da59e432d9014bac7e2d14dc9"
format_stamp: "Formatted at 2026-08-09 22:43:22 on dist-test-slave-chsk"
server_key: "f325a1d55a1b20a70028c6bc0eecf366"
server_key_iv: "e535df36cb4912778409ee0673e512c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:22.140453 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 22:43:22.144037 22601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:22.145571 22584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:43:22.145754 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "1574e54da59e432d9014bac7e2d14dc9"
format_stamp: "Formatted at 2026-08-09 22:43:22 on dist-test-slave-chsk"
server_key: "f325a1d55a1b20a70028c6bc0eecf366"
server_key_iv: "e535df36cb4912778409ee0673e512c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:22.145941 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:22.185844 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:22.189936 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:22.190203 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:22.191128 22584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:22.192476 22584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:22.192559 22584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:22.192649 22584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:22.192694 22584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:22.226814 22584 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:44963
I20260809 22:43:22.226873 22722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:44963 every 8 connection(s)
I20260809 22:43:22.228189 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:22.233476 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22584
I20260809 22:43:22.233672 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:22.234104 21614 external_mini_cluster.cc:1428] Setting key d90f8bff70310a8d2a02ec9624c6d94c
I20260809 22:43:22.236410 22436 heartbeater.cc:503] Master 127.21.27.190:40805 was elected leader, sending a full tablet report...
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:22.248679 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.230780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:56849 (local address 127.21.27.190:40805)
0809 22:43:22.231244 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.231249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.232204 (+ 955us) server_negotiation.cc:407] Connection header received
0809 22:43:22.234145 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.234159 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.234246 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.234363 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.235960 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.237094 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.238786 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.239275 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.239374 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.242637 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.242642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.242645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.242677 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.245476 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.246186 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.246190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.246192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.246314 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.246675 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.246678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.246680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.246999 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.247124 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.247997 (+ 873us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.248276 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":199,"threads_started":1}
I20260809 22:43:22.249868 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.231378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40805 (local address 127.21.27.131:56849)
0809 22:43:22.232044 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.232087 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.233881 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.234582 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.234590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.235194 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:22.235793 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.235810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.237300 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.237305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.238662 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.238674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.239451 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.239456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.239648 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.240276 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:22.240298 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:22.242422 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:22.245585 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:22.245592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:22.245618 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:22.246087 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:22.246403 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:22.246406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:22.246408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:22.246582 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:22.247118 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:22.247123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:22.247547 (+ 424us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.247882 (+ 335us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.248454 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":277,"threads_started":1}
I20260809 22:43:22.251317 22724 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40805
I20260809 22:43:22.251608 22724 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:22.252319 22724 heartbeater.cc:511] Master 127.21.27.190:40805 requested a full tablet report, sending...
I20260809 22:43:22.253633 22232 ts_manager.cc:194] Registered new tserver with Master: 1574e54da59e432d9014bac7e2d14dc9 (127.21.27.131:44963)
I20260809 22:43:22.254536 22232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:56849
I20260809 22:43:22.261827 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:43:22.281189 22726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.268225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.21.27.190:40805)
0809 22:43:22.268471 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.268477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.268787 (+ 310us) server_negotiation.cc:407] Connection header received
0809 22:43:22.268928 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.268932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.269007 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.269086 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.270685 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.271451 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.272845 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.273287 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.273382 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.275675 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.275680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.275684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.275714 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.278507 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.279161 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.279169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.279172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.279311 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.279685 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.279692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.279697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.280025 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.280298 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.280640 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.280869 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:43:22.288254 22232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:22.295416 22232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:22.324443 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.308653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.21.27.131:44963)
0809 22:43:22.316131 (+ 7478us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.316139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.316169 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:43:22.316256 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.316266 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.316466 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.317508 (+ 1042us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:22.317828 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.318860 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.322207 (+ 3347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323108 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323583 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.323813 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.323882 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.324044 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7316,"thread_start_us":7138,"threads_started":1}
I20260809 22:43:22.324586 22742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.310624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40837 (local address 127.0.0.1:36238)
0809 22:43:22.311246 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.311267 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.311398 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.317022 (+ 5624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.317030 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.317066 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:22.317599 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.317614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.319544 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.319557 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.321314 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.321329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323279 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.323514 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.323610 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.323750 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.323992 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":12,"thread_start_us":781,"threads_started":1}
I20260809 22:43:22.324644 22740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.308480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:44963 (local address 127.0.0.1:36188)
0809 22:43:22.309696 (+ 1216us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.309732 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.309892 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.316704 (+ 6812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.316710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.316741 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:22.317272 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.317286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.318990 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.318998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.321970 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.321981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323369 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.323612 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.323663 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.323743 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.324158 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":418,"thread_start_us":1694,"threads_started":1}
I20260809 22:43:22.325177 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.309407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.21.27.129:39043)
0809 22:43:22.315772 (+ 6365us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.315783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.315817 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:43:22.315926 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.315935 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.316220 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.316427 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:22.317215 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.318902 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.322126 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323612 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323761 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.323976 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.324701 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.324872 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6167,"thread_start_us":5995,"threads_started":1}
I20260809 22:43:22.325333 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.311254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.21.27.130:40837)
0809 22:43:22.316450 (+ 5196us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.316459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.316486 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:43:22.316572 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.316581 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.316793 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.317793 (+ 1000us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:22.318002 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.319409 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.321515 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323045 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323218 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.323419 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.324852 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.325026 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5038,"thread_start_us":4933,"threads_started":1}
I20260809 22:43:22.326412 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.308963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39043 (local address 127.0.0.1:59408)
0809 22:43:22.310449 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.310465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.310586 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.316479 (+ 5893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.316486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.316522 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:22.316989 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.317005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.319054 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.319059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.321965 (+ 2906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.321981 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.323976 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.323982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.324177 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.324214 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.324308 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.324443 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1365,"mutex_wait_us":169,"thread_start_us":611,"threads_started":1}
I20260809 22:43:22.328967 22651 tablet_service.cc:1467] Processing CreateTablet for tablet f9a3a4d089d049488bdfa07547cc8412 (DEFAULT_TABLE table=test-table [id=0e4b1d26e516440e96e3674dfb30cd4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:22.328967 22510 tablet_service.cc:1467] Processing CreateTablet for tablet f9a3a4d089d049488bdfa07547cc8412 (DEFAULT_TABLE table=test-table [id=0e4b1d26e516440e96e3674dfb30cd4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:22.329245 22366 tablet_service.cc:1467] Processing CreateTablet for tablet f9a3a4d089d049488bdfa07547cc8412 (DEFAULT_TABLE table=test-table [id=0e4b1d26e516440e96e3674dfb30cd4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:22.330413 22366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a3a4d089d049488bdfa07547cc8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:22.330480 22651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a3a4d089d049488bdfa07547cc8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:22.330480 22510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9a3a4d089d049488bdfa07547cc8412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:22.343912 22744 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Bootstrap starting.
I20260809 22:43:22.344916 22745 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Bootstrap starting.
I20260809 22:43:22.347908 22746 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Bootstrap starting.
I20260809 22:43:22.348786 22745 tablet_bootstrap.cc:654] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:22.350119 22744 tablet_bootstrap.cc:654] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:22.350174 22745 log.cc:824] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:22.351604 22744 log.cc:824] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:22.351840 22746 tablet_bootstrap.cc:654] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:22.353061 22746 log.cc:824] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:22.354871 22744 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: No bootstrap required, opened a new log
I20260809 22:43:22.355113 22744 ts_tablet_manager.cc:1397] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260809 22:43:22.355404 22745 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: No bootstrap required, opened a new log
I20260809 22:43:22.355687 22745 ts_tablet_manager.cc:1397] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 22:43:22.356825 22746 tablet_bootstrap.cc:492] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: No bootstrap required, opened a new log
I20260809 22:43:22.357143 22746 ts_tablet_manager.cc:1397] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 22:43:22.363087 22744 raft_consensus.cc:348] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.363600 22744 raft_consensus.cc:374] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:22.363742 22744 raft_consensus.cc:729] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820604f0ad75413cbe23cb4a53536c4b, State: Initialized, Role: FOLLOWER
I20260809 22:43:22.364313 22744 consensus_queue.cc:260] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.366282 22745 raft_consensus.cc:348] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.366747 22745 raft_consensus.cc:374] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:22.366850 22745 raft_consensus.cc:729] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b1f67b44da4b4ba19017e005ea0b4b, State: Initialized, Role: FOLLOWER
I20260809 22:43:22.367580 22745 consensus_queue.cc:260] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.367686 22746 raft_consensus.cc:348] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.368106 22746 raft_consensus.cc:374] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:22.368237 22746 raft_consensus.cc:729] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1574e54da59e432d9014bac7e2d14dc9, State: Initialized, Role: FOLLOWER
I20260809 22:43:22.369022 22744 ts_tablet_manager.cc:1428] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260809 22:43:22.369081 22746 consensus_queue.cc:260] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.370080 22579 heartbeater.cc:503] Master 127.21.27.190:40805 was elected leader, sending a full tablet report...
I20260809 22:43:22.371090 22745 ts_tablet_manager.cc:1428] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 22:43:22.372115 22724 heartbeater.cc:503] Master 127.21.27.190:40805 was elected leader, sending a full tablet report...
I20260809 22:43:22.372970 22746 ts_tablet_manager.cc:1428] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260809 22:43:22.451421 22580 tablet.cc:2369] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:43:22.456439 22437 tablet.cc:2369] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:43:22.479967 22725 tablet.cc:2369] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:22.511919 22750 raft_consensus.cc:482] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:22.512261 22750 raft_consensus.cc:504] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.513887 22750 leader_election.cc:288] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1574e54da59e432d9014bac7e2d14dc9 (127.21.27.131:44963), 04b1f67b44da4b4ba19017e005ea0b4b (127.21.27.130:40837)
I20260809 22:43:22.520617 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.514300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:44963 (local address 127.21.27.129:57457)
0809 22:43:22.515091 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.515113 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.515290 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.515725 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.515732 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.515759 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:22.516144 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.516153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.517144 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.517151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.518600 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.518611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.520020 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.520024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.520145 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.520172 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.520248 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.520336 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":434,"threads_started":1}
I20260809 22:43:22.520696 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.514304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:57457 (local address 127.21.27.131:44963)
0809 22:43:22.514486 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.514490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.515220 (+ 730us) server_negotiation.cc:407] Connection header received
0809 22:43:22.515464 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.515471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.515563 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.515679 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:22.516292 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.517014 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.518753 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.519810 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.519994 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.520079 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.520294 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.520432 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:43:22.521457 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.515550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:56077 (local address 127.21.27.130:40837)
0809 22:43:22.515770 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.515777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.516062 (+ 285us) server_negotiation.cc:407] Connection header received
0809 22:43:22.516260 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.516264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.516321 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.516414 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:22.516920 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.517585 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.519295 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.520502 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.520662 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.520749 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.521124 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.521275 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:43:22.521672 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.515244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40837 (local address 127.21.27.129:56077)
0809 22:43:22.515967 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:22.515981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:22.516130 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:22.516428 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:22.516432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:22.516455 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:22.516779 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.516787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.517730 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.517735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.519088 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:22.519099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.520751 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.520759 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:22.520929 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.520959 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:43:22.521059 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:43:22.521200 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"spinlock_wait_cycles":19328,"thread_start_us":531,"threads_started":1}
I20260809 22:43:22.522161 22672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a3a4d089d049488bdfa07547cc8412" candidate_uuid: "820604f0ad75413cbe23cb4a53536c4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1574e54da59e432d9014bac7e2d14dc9" is_pre_election: true
I20260809 22:43:22.522656 22672 raft_consensus.cc:2454] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820604f0ad75413cbe23cb4a53536c4b in term 0.
I20260809 22:43:22.522727 22531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a3a4d089d049488bdfa07547cc8412" candidate_uuid: "820604f0ad75413cbe23cb4a53536c4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" is_pre_election: true
I20260809 22:43:22.523268 22531 raft_consensus.cc:2454] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820604f0ad75413cbe23cb4a53536c4b in term 0.
I20260809 22:43:22.523427 22314 leader_election.cc:302] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1574e54da59e432d9014bac7e2d14dc9, 820604f0ad75413cbe23cb4a53536c4b; no voters:
I20260809 22:43:22.523825 22750 raft_consensus.cc:2781] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:22.523983 22750 raft_consensus.cc:482] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:22.524104 22750 raft_consensus.cc:3037] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:22.526286 22750 raft_consensus.cc:504] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.526876 22750 leader_election.cc:288] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [CANDIDATE]: Term 1 election: Requested vote from peers 1574e54da59e432d9014bac7e2d14dc9 (127.21.27.131:44963), 04b1f67b44da4b4ba19017e005ea0b4b (127.21.27.130:40837)
I20260809 22:43:22.527339 22672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a3a4d089d049488bdfa07547cc8412" candidate_uuid: "820604f0ad75413cbe23cb4a53536c4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1574e54da59e432d9014bac7e2d14dc9"
I20260809 22:43:22.527406 22531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9a3a4d089d049488bdfa07547cc8412" candidate_uuid: "820604f0ad75413cbe23cb4a53536c4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04b1f67b44da4b4ba19017e005ea0b4b"
I20260809 22:43:22.527546 22672 raft_consensus.cc:3037] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:22.527638 22531 raft_consensus.cc:3037] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:22.529835 22672 raft_consensus.cc:2454] T f9a3a4d089d049488bdfa07547cc8412 P 1574e54da59e432d9014bac7e2d14dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820604f0ad75413cbe23cb4a53536c4b in term 1.
I20260809 22:43:22.529853 22531 raft_consensus.cc:2454] T f9a3a4d089d049488bdfa07547cc8412 P 04b1f67b44da4b4ba19017e005ea0b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820604f0ad75413cbe23cb4a53536c4b in term 1.
I20260809 22:43:22.530195 22314 leader_election.cc:302] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1574e54da59e432d9014bac7e2d14dc9, 820604f0ad75413cbe23cb4a53536c4b; no voters:
I20260809 22:43:22.530514 22750 raft_consensus.cc:2781] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:22.530977 22750 raft_consensus.cc:686] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [term 1 LEADER]: Becoming Leader. State: Replica: 820604f0ad75413cbe23cb4a53536c4b, State: Running, Role: LEADER
I20260809 22:43:22.531481 22750 consensus_queue.cc:237] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } }
I20260809 22:43:22.536237 22232 catalog_manager.cc:5495] T f9a3a4d089d049488bdfa07547cc8412 P 820604f0ad75413cbe23cb4a53536c4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 820604f0ad75413cbe23cb4a53536c4b (127.21.27.129). New cstate: current_term: 1 leader_uuid: "820604f0ad75413cbe23cb4a53536c4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1574e54da59e432d9014bac7e2d14dc9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04b1f67b44da4b4ba19017e005ea0b4b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "820604f0ad75413cbe23cb4a53536c4b" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39043 } health_report { overall_health: HEALTHY } } }
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:22.564641 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.549524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.21.27.129:39043)
0809 22:43:22.549762 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.549770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.550347 (+ 577us) server_negotiation.cc:407] Connection header received
0809 22:43:22.550561 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.550566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.550632 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.550722 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.552255 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.552994 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.554206 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.554634 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.554732 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.558929 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.558945 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.558961 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.559002 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.561799 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.562531 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.562538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.562540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.562723 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.563080 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.563086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.563091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.563519 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.563748 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.564086 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.564333 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:22.615711 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.601163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.21.27.129:39043)
0809 22:43:22.601373 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.601382 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.601723 (+ 341us) server_negotiation.cc:407] Connection header received
0809 22:43:22.601936 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.601940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.602019 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.602124 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.603603 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.604348 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.605838 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.606273 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.606409 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.610404 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.610409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.610412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.610449 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.613034 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.613680 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.613687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.613691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.613836 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.614248 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.614257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.614261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.614534 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.614845 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.615097 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.615368 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:22.664486 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.650481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.21.27.129:39043)
0809 22:43:22.650718 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.650728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.651177 (+ 449us) server_negotiation.cc:407] Connection header received
0809 22:43:22.651389 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.651393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.651474 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.651587 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.652968 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.653770 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.655187 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.655631 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.655731 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.659788 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.659793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.659796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.659830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.662187 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.662760 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.662764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.662766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.662879 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.663253 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.663256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.663258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.663566 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.663814 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.663978 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.664195 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 22:43:22.666211 22343 tablet_service.cc:2908] Rejecting scan request for tablet f9a3a4d089d049488bdfa07547cc8412: Uninitialized: clean time has not yet been initialized
I20260809 22:43:22.669157 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22296
I20260809 22:43:22.682008 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.670211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.21.27.129:39043)
0809 22:43:22.670353 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.670357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.670748 (+ 391us) server_negotiation.cc:407] Connection header received
0809 22:43:22.670906 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.670909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.670964 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.671066 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.672271 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.672974 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.674281 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.674698 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.674787 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.676854 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.676858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.676861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.676890 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.679757 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.680463 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.680467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.680469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.680586 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.680907 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.680910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.680912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.681162 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.681342 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.681564 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.681750 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:43:22.682641 22431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:22.773780 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22296
I20260809 22:43:22.792315 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22442
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:22.807313 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.793176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.21.27.130:40837)
0809 22:43:22.793373 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.793379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.793400 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:22.793537 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.793542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.793615 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.793720 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.794908 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.795833 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.797099 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.797514 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.797646 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.801756 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.801768 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.801784 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.801816 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.804529 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.805198 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.805204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.805207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.805336 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.805705 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.805710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.805713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.806089 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.806421 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.806687 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.806909 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:43:22.807993 22574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:22.895459 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22442
I20260809 22:43:22.914064 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22584
Aug 09 22:43:22 dist-test-slave-chsk krb5kdc[22181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:22.928831 22739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:22.914851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.21.27.131:44963)
0809 22:43:22.915050 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:22.915056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:22.915093 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:43:22.915283 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:22.915288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:22.915353 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:22.915456 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:22.916733 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.917429 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.918707 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:22.919084 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:22.919258 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:22.923357 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:22.923370 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:22.923387 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:22.923422 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:22.926158 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.926734 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.926741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.926743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.926862 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:22.927258 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:22.927265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:22.927269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:22.927665 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:22.927954 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:22.928251 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:43:22.928499 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:43:22.929425 22718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:23.016216 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22584
I20260809 22:43:23.034008 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22195
I20260809 22:43:23.035328 22265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:23.146431 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22195
2026-08-09T22:43:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4302 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:23 dist-test-slave-chsk krb5kdc[22774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:23 dist-test-slave-chsk krb5kdc[22774](info): set up 2 sockets
Aug 09 22:43:23 dist-test-slave-chsk krb5kdc[22774](info): commencing operation
krb5kdc: starting...
W20260809 22:43:24.282606 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:24 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:24.490914 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34895
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:24.813620 22787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:24.813961 22787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:24.814119 22787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:24.821385 22787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:24.821521 22787 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:43:24.821616 22787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:24.821731 22787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:24.821815 22787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:24.821906 22787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:24.831681 22787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:34895
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:24.832995 22787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:24.834272 22787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:24.845911 22792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:24.845969 22795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:24.845880 22793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:24.846552 22787 server_base.cc:1034] running on GCE node
I20260809 22:43:24.847560 22787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:24.848882 22787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:24.850139 22787 hybrid_clock.cc:648] HybridClock initialized: now 1786315404850076 us; error 40 us; skew 500 ppm
I20260809 22:43:24.850494 22787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:24 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315404, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:24.853717 22787 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:24.855329 22787 webserver.cc:460] Webserver started at http://127.21.27.190:35457/ using document root <none> and password file <none>
I20260809 22:43:24.855975 22787 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:24.856062 22787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:24.856343 22787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:24.859207 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "91e2f191227648b8a88933ab83e49be3"
format_stamp: "Formatted at 2026-08-09 22:43:24 on dist-test-slave-chsk"
server_key: "c4ec1801cb3b7925a595ee8046c2925a"
server_key_iv: "501b18587b6eec1080a5e9bde58abf8b"
server_key_version: "encryptionkey@0"
I20260809 22:43:24.860010 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "91e2f191227648b8a88933ab83e49be3"
format_stamp: "Formatted at 2026-08-09 22:43:24 on dist-test-slave-chsk"
server_key: "c4ec1801cb3b7925a595ee8046c2925a"
server_key_iv: "501b18587b6eec1080a5e9bde58abf8b"
server_key_version: "encryptionkey@0"
I20260809 22:43:24.865026 22787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:43:24.868263 22804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:24.869539 22787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:43:24.869673 22787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "91e2f191227648b8a88933ab83e49be3"
format_stamp: "Formatted at 2026-08-09 22:43:24 on dist-test-slave-chsk"
server_key: "c4ec1801cb3b7925a595ee8046c2925a"
server_key_iv: "501b18587b6eec1080a5e9bde58abf8b"
server_key_version: "encryptionkey@0"
I20260809 22:43:24.869884 22787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:24.888394 22787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:24.892194 22787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:24.892412 22787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:24.914278 22787 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:34895
I20260809 22:43:24.914283 22872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:34895 every 8 connection(s)
I20260809 22:43:24.915699 22787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:24.919540 22873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:24.921047 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22787
I20260809 22:43:24.921312 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:24.921989 21614 external_mini_cluster.cc:1428] Setting key eec6322be111530f8fbfc4aa6ce8b870
I20260809 22:43:24.933576 22873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Bootstrap starting.
Aug 09 22:43:24 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:24.937373 22873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:24.938603 22873 log.cc:824] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:24.941380 22873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: No bootstrap required, opened a new log
I20260809 22:43:24.944757 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:24.923951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.21.27.190:34895)
0809 22:43:24.924793 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:24.924805 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:24.924843 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:43:24.926452 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:24.926474 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:24.926838 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:24.927228 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:24.928951 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:24.930197 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:24.931970 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:24.932604 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:24.932725 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:24.937504 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:24.937528 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:24.937547 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:24.937610 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:24.940776 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:24.941488 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:24.941498 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:24.941506 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:24.941694 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:24.942108 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:24.942112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:24.942114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:24.942645 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:24.942979 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:24.943414 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 22:43:24.943819 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":424,"threads_started":1}
I20260809 22:43:24.949286 22873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } }
I20260809 22:43:24.949645 22873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:24.949759 22873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e2f191227648b8a88933ab83e49be3, State: Initialized, Role: FOLLOWER
I20260809 22:43:24.950348 22873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } }
I20260809 22:43:24.950609 22873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:24.950778 22873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:24.950953 22873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:24.953107 22873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } }
I20260809 22:43:24.953624 22873 leader_election.cc:302] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91e2f191227648b8a88933ab83e49be3; no voters:
I20260809 22:43:24.954180 22873 leader_election.cc:288] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:24.954329 22878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:24.955076 22878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [term 1 LEADER]: Becoming Leader. State: Replica: 91e2f191227648b8a88933ab83e49be3, State: Running, Role: LEADER
I20260809 22:43:24.955655 22878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } }
I20260809 22:43:24.956262 22873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:24.958271 22880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91e2f191227648b8a88933ab83e49be3. Latest consensus state: current_term: 1 leader_uuid: "91e2f191227648b8a88933ab83e49be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } } }
I20260809 22:43:24.958639 22880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:24.959196 22879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91e2f191227648b8a88933ab83e49be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e2f191227648b8a88933ab83e49be3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34895 } } }
I20260809 22:43:24.959625 22879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:24.964184 22883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:24.968963 22883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:24.977384 22883 catalog_manager.cc:1347] Generated new cluster ID: 77a51ddaea78424cbb7bf513ed23e30b
I20260809 22:43:24.977496 22883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:24.994563 22883 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:24.996295 22883 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:25.002794 22883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Generated new TSK 0
I20260809 22:43:25.003729 22883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:25.054978 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34895
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:25.385502 22900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:25.385775 22900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:25.385950 22900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:25.393000 22900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:25.393126 22900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:25.393291 22900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:25.393402 22900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:25.402910 22900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:25.404119 22900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:25.405408 22900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:25.418015 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:25.417984 22905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:25.418080 22906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:25.418941 22900 server_base.cc:1034] running on GCE node
I20260809 22:43:25.419595 22900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:25.420462 22900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:25.421698 22900 hybrid_clock.cc:648] HybridClock initialized: now 1786315405421645 us; error 34 us; skew 500 ppm
I20260809 22:43:25.421986 22900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:25 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:25.425237 22900 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:25.426508 22900 webserver.cc:460] Webserver started at http://127.21.27.129:42387/ using document root <none> and password file <none>
I20260809 22:43:25.427134 22900 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:25.427289 22900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:25.427559 22900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:25.430277 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "33cf6f19101a4839808ae15bc7887d89"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "1c44fc639753ff5dcfee44606ff406ba"
server_key_iv: "beeab2ca05a0c821ed54c3a9facbeeea"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.431027 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "33cf6f19101a4839808ae15bc7887d89"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "1c44fc639753ff5dcfee44606ff406ba"
server_key_iv: "beeab2ca05a0c821ed54c3a9facbeeea"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.435927 22900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:43:25.439299 22915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:25.440583 22900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:43:25.440754 22900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "33cf6f19101a4839808ae15bc7887d89"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "1c44fc639753ff5dcfee44606ff406ba"
server_key_iv: "beeab2ca05a0c821ed54c3a9facbeeea"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.440964 22900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:25.458400 22900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:25.462785 22900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:25.463071 22900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:25.465967 22900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:25.466078 22900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:25.466214 22900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:25.466300 22900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:43:25 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:25.489486 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.466547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48175 (local address 127.21.27.190:34895)
0809 22:43:25.468927 (+ 2380us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:25.468935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:25.468961 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:43:25.470608 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:25.470615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:25.470717 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:25.470861 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:25.473235 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.474315 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.477561 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.478019 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.478117 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.483472 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:25.483478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:25.483481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:25.483514 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:25.486200 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:25.486942 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:25.486947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:25.486949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:25.487064 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:25.487539 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:25.487544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:25.487548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:25.487827 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:25.487977 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:25.488895 (+ 918us) server_negotiation.cc:299] Negotiation successful
0809 22:43:25.489114 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2233,"thread_start_us":2134,"threads_started":1}
I20260809 22:43:25.491034 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.467206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.129:48175)
0809 22:43:25.468276 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:25.468351 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:25.470311 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:25.471196 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:25.471206 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:25.471999 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:25.472987 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.473018 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.474531 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.474540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.477382 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.477407 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.478253 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.478262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.478656 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.479594 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:25.479619 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:25.483222 (+ 3603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:25.486351 (+ 3129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:25.486360 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:25.486375 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:25.486806 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:25.487194 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:25.487201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:25.487205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:25.487410 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:25.487975 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:25.487983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:25.488440 (+ 457us) client_negotiation.cc:769] Sending connection context
0809 22:43:25.488802 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 22:43:25.489362 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":405,"threads_started":1}
I20260809 22:43:25.516172 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.497946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.129:59245)
0809 22:43:25.499157 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:25.499187 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:25.499362 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:25.499816 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:25.499821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:25.500670 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:25.501322 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.501335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.502767 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.502774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.504246 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.504269 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.505339 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.505349 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.505600 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.506427 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:25.506454 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:25.507950 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:25.512313 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:25.512321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:25.512329 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:25.512944 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:25.513473 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:25.513481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:25.513488 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:25.513697 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:25.514564 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:25.514572 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:25.514858 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 22:43:25.515072 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 22:43:25.515598 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":755,"threads_started":1}
I20260809 22:43:25.516649 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.498032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:59245 (local address 127.21.27.190:34895)
0809 22:43:25.498289 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:25.498300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:25.499298 (+ 998us) server_negotiation.cc:407] Connection header received
0809 22:43:25.499560 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:25.499569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:25.499670 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:25.499788 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:25.501496 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.502608 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.504413 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.505103 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.505245 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.508223 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:25.508230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:25.508235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:25.508272 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:25.512152 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:25.513111 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:25.513120 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:25.513125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:25.513302 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:25.513868 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:25.513877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:25.513882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:25.514300 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:25.514477 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:25.515945 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0809 22:43:25.516193 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:43:25.520551 22900 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:37609
I20260809 22:43:25.520596 23040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:37609 every 8 connection(s)
I20260809 22:43:25.521919 22900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:25.524085 23041 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34895
I20260809 22:43:25.524339 23041 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:25.524994 23041 heartbeater.cc:511] Master 127.21.27.190:34895 requested a full tablet report, sending...
I20260809 22:43:25.526242 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 22900
I20260809 22:43:25.526486 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:25.527225 21614 external_mini_cluster.cc:1428] Setting key 366ed649bd79d577e5c46e4a45de2c90
I20260809 22:43:25.527114 22824 ts_manager.cc:194] Registered new tserver with Master: 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609)
I20260809 22:43:25.529551 22824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:48175
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:25.571878 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34895
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:25.892952 23045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:25.893285 23045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:25.893455 23045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:25.900683 23045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:25.900844 23045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:25.901026 23045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:25.901176 23045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:25.910854 23045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:25.912209 23045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:25.913687 23045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:25.927786 23050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:25.928344 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:25.929793 23053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:25.929971 23045 server_base.cc:1034] running on GCE node
I20260809 22:43:25.930595 23045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:25.931622 23045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:25.932873 23045 hybrid_clock.cc:648] HybridClock initialized: now 1786315405932798 us; error 52 us; skew 500 ppm
I20260809 22:43:25.933192 23045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:25 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:25.940166 23045 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:25.941799 23045 webserver.cc:460] Webserver started at http://127.21.27.130:41201/ using document root <none> and password file <none>
I20260809 22:43:25.942799 23045 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:25.942912 23045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:25.943317 23045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:25.946280 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "7652638b9e34422da8d1d818ee196447"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "a3b6ce0113b9b8c46a08db40f24747cf"
server_key_iv: "d6f0de20b176bf202652100d7f825f63"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.947089 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "7652638b9e34422da8d1d818ee196447"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "a3b6ce0113b9b8c46a08db40f24747cf"
server_key_iv: "d6f0de20b176bf202652100d7f825f63"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.952349 23045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:43:25.955686 23060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:25.957108 23045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:43:25.957283 23045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "7652638b9e34422da8d1d818ee196447"
format_stamp: "Formatted at 2026-08-09 22:43:25 on dist-test-slave-chsk"
server_key: "a3b6ce0113b9b8c46a08db40f24747cf"
server_key_iv: "d6f0de20b176bf202652100d7f825f63"
server_key_version: "encryptionkey@0"
I20260809 22:43:25.957496 23045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:25.971980 23045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:25.976035 23045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:25.976306 23045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:25.979038 23045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:25.979164 23045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:25.979329 23045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:25.979409 23045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:43:25 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:26.004534 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.979827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:37081 (local address 127.21.27.190:34895)
0809 22:43:25.980013 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:25.980019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:25.981446 (+ 1427us) server_negotiation.cc:407] Connection header received
0809 22:43:25.983542 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:25.983549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:25.983658 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:25.983778 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:25.986015 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.987380 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.990662 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.991165 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.991306 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.996773 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:25.996782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:25.996790 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:25.996840 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:26.000769 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.001799 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.001807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.001811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.001946 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.002449 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.002455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.002459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.002763 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:26.002930 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.003933 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.004178 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:43:26.005935 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:25.980376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.130:37081)
0809 22:43:25.981252 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:25.981300 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:25.983230 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:25.984084 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:25.984094 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:25.984900 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:25.985792 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.985822 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.987620 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.987625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.990490 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:25.990504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:25.991562 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:25.991574 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:25.991768 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:25.992505 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:25.992548 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:25.996495 (+ 3947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:26.000951 (+ 4456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.000957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.000971 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.001662 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.002104 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.002109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.002113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.002328 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.002939 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:26.002950 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:26.003490 (+ 540us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.003846 (+ 356us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.004436 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":384,"threads_started":1}
I20260809 22:43:26.026746 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.011595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.130:33005)
0809 22:43:26.012168 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.012185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.012354 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.012713 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.012718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.013269 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:26.013564 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.013575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.014763 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.014772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.016157 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.016172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.017105 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.017114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.017376 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.018300 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:26.018334 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:26.019759 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:26.023533 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.023542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.023551 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.024248 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.024805 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.024813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.024820 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.025007 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.025871 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:26.025877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:26.026037 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.026160 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.026396 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":321,"threads_started":1}
I20260809 22:43:26.027120 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.011560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:33005 (local address 127.21.27.190:34895)
0809 22:43:26.011731 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.011735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.012268 (+ 533us) server_negotiation.cc:407] Connection header received
0809 22:43:26.012511 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.012515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.012583 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.012686 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:26.013719 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.014634 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.016386 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.016874 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.016988 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.020064 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:26.020072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:26.020079 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:26.020126 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:26.023409 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.024445 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.024451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.024454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.024636 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.025202 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.025211 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.025216 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.025724 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:26.025916 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.026342 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.026651 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:43:26.035508 23045 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:40297
I20260809 22:43:26.035545 23190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:40297 every 8 connection(s)
I20260809 22:43:26.036682 23045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:26.038580 23191 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34895
I20260809 22:43:26.038820 23191 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:26.039492 23191 heartbeater.cc:511] Master 127.21.27.190:34895 requested a full tablet report, sending...
I20260809 22:43:26.040750 22824 ts_manager.cc:194] Registered new tserver with Master: 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297)
I20260809 22:43:26.041489 22824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:37081
I20260809 22:43:26.042362 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23045
I20260809 22:43:26.042593 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:26.043031 21614 external_mini_cluster.cc:1428] Setting key 899ce42b399392ee4022f16ad86d6de5
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:26.085911 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34895
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:26.410451 23195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:26.410724 23195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:26.410840 23195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:26.417753 23195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:26.417852 23195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:26.417932 23195 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:26.418000 23195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:26.427258 23195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:26.428408 23195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:26.429654 23195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:26.442723 23200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:26.442818 23203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:26.442781 23201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:26.443506 23195 server_base.cc:1034] running on GCE node
I20260809 22:43:26.444236 23195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:26.445147 23195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:26.446355 23195 hybrid_clock.cc:648] HybridClock initialized: now 1786315406446294 us; error 37 us; skew 500 ppm
I20260809 22:43:26.446607 23195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:26 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:26.449790 23195 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:26.451254 23195 webserver.cc:460] Webserver started at http://127.21.27.131:37149/ using document root <none> and password file <none>
I20260809 22:43:26.451874 23195 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:26.451963 23195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:26.452235 23195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:26.455328 23195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "3282cdffca104c308570d41897afa394"
format_stamp: "Formatted at 2026-08-09 22:43:26 on dist-test-slave-chsk"
server_key: "25312eebd969da12f5f8f406a6554ad2"
server_key_iv: "ae451af347c1b95de536cc27cb793dfd"
server_key_version: "encryptionkey@0"
I20260809 22:43:26.456115 23195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "3282cdffca104c308570d41897afa394"
format_stamp: "Formatted at 2026-08-09 22:43:26 on dist-test-slave-chsk"
server_key: "25312eebd969da12f5f8f406a6554ad2"
server_key_iv: "ae451af347c1b95de536cc27cb793dfd"
server_key_version: "encryptionkey@0"
I20260809 22:43:26.461315 23195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:26.464841 23210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:26.466089 23195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:43:26.466244 23195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "3282cdffca104c308570d41897afa394"
format_stamp: "Formatted at 2026-08-09 22:43:26 on dist-test-slave-chsk"
server_key: "25312eebd969da12f5f8f406a6554ad2"
server_key_iv: "ae451af347c1b95de536cc27cb793dfd"
server_key_version: "encryptionkey@0"
I20260809 22:43:26.466464 23195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:26.491745 23195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:26.496307 23195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:26.496592 23195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:26.499188 23195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:26.499296 23195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:26.499420 23195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:26.499495 23195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:43:26 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:26.527213 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.500553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.131:53285)
0809 22:43:26.501346 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.501392 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.503378 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.504309 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.504326 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.505101 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:26.505929 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.505951 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.507876 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.507883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.510256 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.510268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.511811 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.511822 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.512161 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.512984 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:26.513006 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:26.516415 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:26.521558 (+ 5143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.521567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.521581 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.522215 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.522732 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.522737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.522743 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.522981 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.523799 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:26.523811 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:26.524400 (+ 589us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.524751 (+ 351us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.525438 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":356,"threads_started":1}
I20260809 22:43:26.528393 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.499865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:53285 (local address 127.21.27.190:34895)
0809 22:43:26.500075 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.500084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.501523 (+ 1439us) server_negotiation.cc:407] Connection header received
0809 22:43:26.503703 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.503710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.503799 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.503922 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:26.506208 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.507684 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.510409 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.511030 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.511201 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.516735 (+ 5534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:26.516746 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:26.516751 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:26.516803 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:26.521369 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.522351 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.522360 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.522367 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.522591 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.523101 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.523109 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.523115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.523602 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:26.527724 (+ 4122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.527893 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.528152 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:43:26.532446 23041 heartbeater.cc:503] Master 127.21.27.190:34895 was elected leader, sending a full tablet report...
I20260809 22:43:26.548667 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.534832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.21.27.131:34971)
0809 22:43:26.535499 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.535518 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.535642 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.535982 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.535986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.536455 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:26.536734 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.536742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.537752 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.537757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.538892 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.538914 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.540050 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.540056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.540383 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.541251 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:26.541282 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:26.542567 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:26.546035 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.546040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.546044 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.546427 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.546806 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:26.546810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:26.546814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:26.546951 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:26.547571 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:26.547576 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:26.547796 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.547980 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.548340 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":375,"threads_started":1}
I20260809 22:43:26.548926 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.535312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:34971 (local address 127.21.27.190:34895)
0809 22:43:26.535583 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.535589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.535607 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:43:26.535785 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.535789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.535857 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.535959 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:26.536889 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.537579 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.539116 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.539792 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.539941 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.542828 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:26.542835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:26.542841 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:26.542884 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:26.545900 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.546554 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.546559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.546563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.546691 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.547060 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.547066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.547070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.547444 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:26.547608 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.548126 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.548418 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260809 22:43:26.560563 23195 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45011
I20260809 22:43:26.560662 23335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45011 every 8 connection(s)
I20260809 22:43:26.561961 23195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:26.564258 23336 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34895
I20260809 22:43:26.564452 23336 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:26.565025 23336 heartbeater.cc:511] Master 127.21.27.190:34895 requested a full tablet report, sending...
I20260809 22:43:26.565522 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23195
I20260809 22:43:26.565742 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:26.566262 21614 external_mini_cluster.cc:1428] Setting key 0f1b04c1f343f038dfd2de2c8c7f60f8
I20260809 22:43:26.566633 22824 ts_manager.cc:194] Registered new tserver with Master: 3282cdffca104c308570d41897afa394 (127.21.27.131:45011)
I20260809 22:43:26.567786 22824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:53285
I20260809 22:43:26.569296 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:43:26 dist-test-slave-chsk krb5kdc[22774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:43:26 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:26.622249 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.608467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.21.27.190:34895)
0809 22:43:26.608661 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.608666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.608981 (+ 315us) server_negotiation.cc:407] Connection header received
0809 22:43:26.609130 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.609134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.609213 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.609307 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:26.610662 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.611543 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.612787 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.613202 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.613314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.617273 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:26.617278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:26.617281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:26.617315 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:26.619945 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.620512 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.620516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.620519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.620649 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:26.620965 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:26.620968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:26.620970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:26.621220 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:26.621397 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.621643 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.621803 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:43:26.626477 22824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:26.633280 22824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:26.656126 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.645217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45011 (local address 127.0.0.1:51656)
0809 22:43:26.646261 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.646308 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.646513 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.647807 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.647824 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.647863 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.648330 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.648343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.650445 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.650450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.652920 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.652934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.655061 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.655068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.655290 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.655349 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.655482 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.655690 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":582,"threads_started":1}
I20260809 22:43:26.657044 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.645482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.21.27.131:45011)
0809 22:43:26.646994 (+ 1512us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.647007 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.647049 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:43:26.647225 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.647237 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.647535 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.647767 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.648594 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.650278 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.653108 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.654552 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.654691 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.654926 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.656517 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.656713 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1298,"thread_start_us":1106,"threads_started":1}
I20260809 22:43:26.657291 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.645885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.21.27.130:40297)
0809 22:43:26.647584 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.647590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.648432 (+ 842us) server_negotiation.cc:407] Connection header received
0809 22:43:26.648730 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.648738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.648957 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.649153 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.649984 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.651432 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.653930 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.655302 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.655425 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.655615 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.656649 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.656982 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1561,"thread_start_us":1439,"threads_started":1}
I20260809 22:43:26.656821 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.645473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40297 (local address 127.0.0.1:58466)
0809 22:43:26.648042 (+ 2569us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.648059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.648541 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.649368 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.649373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.649396 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.649805 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.649817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.651603 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.651610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.653742 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.653757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.655763 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.655769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.655956 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.655985 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.656074 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.656229 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2418,"mutex_wait_us":1,"thread_start_us":2254,"threads_started":1}
I20260809 22:43:26.661520 23265 tablet_service.cc:1467] Processing CreateTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 (DEFAULT_TABLE table=test-table [id=b401f96dd9b34758a9770263bf06b2b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.661906 23113 tablet_service.cc:1467] Processing CreateTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 (DEFAULT_TABLE table=test-table [id=b401f96dd9b34758a9770263bf06b2b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.663189 23265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65a7f4be0494b1f9fa09fe8ec94b930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.663225 23113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65a7f4be0494b1f9fa09fe8ec94b930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.667193 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.648156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37609 (local address 127.0.0.1:43706)
0809 22:43:26.657635 (+ 9479us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.657691 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.657794 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.659578 (+ 1784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.659583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.659611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.659802 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.659813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.662663 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.662669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.664543 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.664558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.666414 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.666422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.666615 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.666651 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.666720 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.666871 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9388,"thread_start_us":10352,"threads_started":1}
I20260809 22:43:26.668213 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.648479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.21.27.129:37609)
0809 22:43:26.658426 (+ 9947us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.658435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.658461 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:43:26.658553 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.658559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.659366 (+ 807us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.659579 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.661000 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.662494 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.664787 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.666221 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.667465 (+ 1244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.667676 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.667749 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.667916 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9776,"thread_start_us":9610,"threads_started":1}
I20260809 22:43:26.671835 22968 tablet_service.cc:1467] Processing CreateTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 (DEFAULT_TABLE table=test-table [id=b401f96dd9b34758a9770263bf06b2b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.673228 22968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65a7f4be0494b1f9fa09fe8ec94b930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.676977 23356 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Bootstrap starting.
I20260809 22:43:26.679935 23357 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Bootstrap starting.
I20260809 22:43:26.680383 23356 tablet_bootstrap.cc:654] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.681807 23356 log.cc:824] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:26.683346 23357 tablet_bootstrap.cc:654] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.684548 23357 log.cc:824] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:26.685038 23356 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: No bootstrap required, opened a new log
I20260809 22:43:26.685354 23356 ts_tablet_manager.cc:1397] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 22:43:26.687551 23357 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: No bootstrap required, opened a new log
I20260809 22:43:26.687666 23360 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Bootstrap starting.
I20260809 22:43:26.687935 23357 ts_tablet_manager.cc:1397] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.006s
I20260809 22:43:26.691183 23360 tablet_bootstrap.cc:654] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.692193 23360 log.cc:824] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:26.694936 23360 tablet_bootstrap.cc:492] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: No bootstrap required, opened a new log
I20260809 22:43:26.695250 23360 ts_tablet_manager.cc:1397] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 22:43:26.695920 23356 raft_consensus.cc:348] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.696344 23356 raft_consensus.cc:374] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.696444 23356 raft_consensus.cc:729] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7652638b9e34422da8d1d818ee196447, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.697034 23356 consensus_queue.cc:260] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.698801 23357 raft_consensus.cc:348] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.699363 23357 raft_consensus.cc:374] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.699474 23357 raft_consensus.cc:729] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3282cdffca104c308570d41897afa394, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.700011 23191 heartbeater.cc:503] Master 127.21.27.190:34895 was elected leader, sending a full tablet report...
I20260809 22:43:26.700142 23357 consensus_queue.cc:260] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.700627 23356 ts_tablet_manager.cc:1428] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260809 22:43:26.701288 23336 heartbeater.cc:503] Master 127.21.27.190:34895 was elected leader, sending a full tablet report...
I20260809 22:43:26.701828 23357 ts_tablet_manager.cc:1428] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 22:43:26.705988 23360 raft_consensus.cc:348] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.706380 23360 raft_consensus.cc:374] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.706482 23360 raft_consensus.cc:729] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33cf6f19101a4839808ae15bc7887d89, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.707198 23360 consensus_queue.cc:260] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.709034 23360 ts_tablet_manager.cc:1428] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260809 22:43:26.768636 23362 raft_consensus.cc:482] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:26.769012 23362 raft_consensus.cc:504] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.770713 23362 leader_election.cc:288] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3282cdffca104c308570d41897afa394 (127.21.27.131:45011), 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609)
W20260809 22:43:26.774371 23042 tablet.cc:2369] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:26.777779 23364 raft_consensus.cc:482] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:26.778035 23364 raft_consensus.cc:504] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.778826 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.771207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:53379 (local address 127.21.27.129:37609)
0809 22:43:26.771443 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.771449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.772337 (+ 888us) server_negotiation.cc:407] Connection header received
0809 22:43:26.772547 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.772552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.772633 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.772741 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.773701 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.774843 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.776827 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.777646 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.777827 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.777912 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.778447 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.778594 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:43:26.778970 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.771205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:45267 (local address 127.21.27.131:45011)
0809 22:43:26.771435 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.771442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.772333 (+ 891us) server_negotiation.cc:407] Connection header received
0809 22:43:26.772547 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.772553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.772658 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.772745 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.773680 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.774799 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.776890 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.777818 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.777979 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.778054 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.778594 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.778736 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:43:26.778962 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.771033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37609 (local address 127.21.27.130:53379)
0809 22:43:26.772174 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.772198 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.772398 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.772781 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.772786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.772818 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.773544 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.773557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.774994 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.774999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.776653 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.776665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.777881 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.777890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.778208 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.778267 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.778407 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.778550 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":57,"spinlock_wait_cycles":14848,"thread_start_us":603,"threads_started":1}
I20260809 22:43:26.779799 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.771206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45011 (local address 127.21.27.130:45267)
0809 22:43:26.772203 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.772217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.772398 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.772823 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.772828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.772849 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.773452 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.773471 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.774954 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.774959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.776737 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.776750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.778207 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.778212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.778417 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.778445 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.779229 (+ 784us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.779351 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":91,"spinlock_wait_cycles":27264,"thread_start_us":581,"threads_started":1}
I20260809 22:43:26.780221 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cf6f19101a4839808ae15bc7887d89" is_pre_election: true
I20260809 22:43:26.780737 22994 raft_consensus.cc:2454] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 0.
I20260809 22:43:26.781479 23061 leader_election.cc:302] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33cf6f19101a4839808ae15bc7887d89, 7652638b9e34422da8d1d818ee196447; no voters:
I20260809 22:43:26.781910 23362 raft_consensus.cc:2781] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:26.782089 23362 raft_consensus.cc:482] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:26.782215 23362 raft_consensus.cc:3037] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:26.782853 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394" is_pre_election: true
I20260809 22:43:26.783427 23364 leader_election.cc:288] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3282cdffca104c308570d41897afa394 (127.21.27.131:45011), 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297)
I20260809 22:43:26.783529 23290 raft_consensus.cc:2454] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 0.
I20260809 22:43:26.785344 23362 raft_consensus.cc:504] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.785993 23362 leader_election.cc:288] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 election: Requested vote from peers 3282cdffca104c308570d41897afa394 (127.21.27.131:45011), 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609)
I20260809 22:43:26.786327 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394"
I20260809 22:43:26.786520 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cf6f19101a4839808ae15bc7887d89"
I20260809 22:43:26.786556 23290 raft_consensus.cc:3037] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:26.786746 22994 raft_consensus.cc:3037] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:43:26.788885 23192 tablet.cc:2369] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:26.789934 23290 raft_consensus.cc:2454] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 1.
I20260809 22:43:26.790210 22994 raft_consensus.cc:2454] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 1.
I20260809 22:43:26.790393 23062 leader_election.cc:302] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3282cdffca104c308570d41897afa394, 7652638b9e34422da8d1d818ee196447; no voters:
I20260809 22:43:26.790771 23362 raft_consensus.cc:2781] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:26.792030 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.781490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48889 (local address 127.21.27.131:45011)
0809 22:43:26.781689 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.781695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.784089 (+ 2394us) server_negotiation.cc:407] Connection header received
0809 22:43:26.784356 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.784362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.784429 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.784483 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.785273 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.786448 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.789414 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.790932 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.791047 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.791098 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.791641 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.791763 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:43:26.792114 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.780396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45011 (local address 127.21.27.129:48889)
0809 22:43:26.783980 (+ 3584us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.783999 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.784201 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.784672 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.784677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.784701 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.785114 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.785130 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.786635 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.786640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.789239 (+ 2599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.789253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.791343 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.791348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.791463 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.791485 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.791617 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.791755 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3448,"thread_start_us":3287,"threads_started":1}
I20260809 22:43:26.792939 23362 raft_consensus.cc:686] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 1 LEADER]: Becoming Leader. State: Replica: 7652638b9e34422da8d1d818ee196447, State: Running, Role: LEADER
I20260809 22:43:26.792907 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394" is_pre_election: true
I20260809 22:43:26.793315 23290 raft_consensus.cc:2379] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33cf6f19101a4839808ae15bc7887d89 in current term 1: Already voted for candidate 7652638b9e34422da8d1d818ee196447 in this term.
I20260809 22:43:26.793478 23362 consensus_queue.cc:237] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } }
I20260809 22:43:26.795207 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.784671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:54125 (local address 127.21.27.130:40297)
0809 22:43:26.784859 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.784866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.784884 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:43:26.785119 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.785127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.785219 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.785286 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:26.787496 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.788552 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.792635 (+ 4083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.793988 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.794161 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.794230 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.794788 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.794909 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:43:26.795243 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.784196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40297 (local address 127.21.27.129:54125)
0809 22:43:26.784789 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:26.784813 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:26.784940 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:26.785491 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:26.785496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:26.785520 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:26.785868 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.785879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.789750 (+ 3871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.789758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.792458 (+ 2700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:26.792479 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.794408 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.794416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:26.794618 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.794652 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:43:26.794763 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 22:43:26.794894 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":783,"threads_started":1}
I20260809 22:43:26.796406 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65a7f4be0494b1f9fa09fe8ec94b930" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7652638b9e34422da8d1d818ee196447" is_pre_election: true
I20260809 22:43:26.797552 22916 leader_election.cc:302] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33cf6f19101a4839808ae15bc7887d89; no voters: 3282cdffca104c308570d41897afa394, 7652638b9e34422da8d1d818ee196447
I20260809 22:43:26.797979 23364 raft_consensus.cc:2726] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:43:26.798399 22823 catalog_manager.cc:5495] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7652638b9e34422da8d1d818ee196447 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "7652638b9e34422da8d1d818ee196447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } health_report { overall_health: UNKNOWN } } }
W20260809 22:43:26.813689 23337 tablet.cc:2369] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:26.827785 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:26.822613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.21.27.130:40297)
0809 22:43:26.822772 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:26.822776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:26.822797 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:26.823092 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:26.823096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:26.823336 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:26.823460 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:26.823790 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.824575 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.825851 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:26.826238 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:26.826333 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:26.826741 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:26.827275 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:26.827374 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:26.827490 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 22:43:26.827554 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:43:26.835659 23290 raft_consensus.cc:1261] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Refusing update from remote peer 7652638b9e34422da8d1d818ee196447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:26.835717 22994 raft_consensus.cc:1261] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Refusing update from remote peer 7652638b9e34422da8d1d818ee196447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:26.836529 23369 consensus_queue.cc:1035] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:26.836936 23362 consensus_queue.cc:1035] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:26.848301 23372 mvcc.cc:205] Tried to move back new op lower bound from 7316747906389659648 to 7316747906230669312. Current Snapshot: MvccSnapshot[applied={T|T < 7316747906389659648}]
I20260809 22:43:26.868209 22824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:26.868511 22824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:26.872207 22824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Sending DeleteTablet for 3 replicas of tablet a65a7f4be0494b1f9fa09fe8ec94b930
I20260809 22:43:26.873453 23265 tablet_service.cc:1514] Processing DeleteTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:26 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:51656
I20260809 22:43:26.873658 23113 tablet_service.cc:1514] Processing DeleteTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:26 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58466
I20260809 22:43:26.873658 22968 tablet_service.cc:1514] Processing DeleteTablet for tablet a65a7f4be0494b1f9fa09fe8ec94b930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:26 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43706
I20260809 22:43:26.875248 22824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:26.876081 22824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:26.885798 22966 tablet_service.cc:1467] Processing CreateTablet for tablet bf65e3322ca44603a9c37eb452f11d8d (DEFAULT_TABLE table=test-table [id=7633dca43a794b7aaf7bf1548ab6d141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.886569 22966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf65e3322ca44603a9c37eb452f11d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.887993 23385 tablet_replica.cc:330] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: stopping tablet replica
I20260809 22:43:26.889102 23385 raft_consensus.cc:2229] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:26.890338 23385 raft_consensus.cc:2258] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:26.891891 23112 tablet_service.cc:1467] Processing CreateTablet for tablet bf65e3322ca44603a9c37eb452f11d8d (DEFAULT_TABLE table=test-table [id=7633dca43a794b7aaf7bf1548ab6d141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.892192 23264 tablet_service.cc:1467] Processing CreateTablet for tablet bf65e3322ca44603a9c37eb452f11d8d (DEFAULT_TABLE table=test-table [id=7633dca43a794b7aaf7bf1548ab6d141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:26.892585 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf65e3322ca44603a9c37eb452f11d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.892871 23264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf65e3322ca44603a9c37eb452f11d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:26.892975 23383 tablet_replica.cc:330] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: stopping tablet replica
I20260809 22:43:26.893546 23383 raft_consensus.cc:2229] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:26.894045 23383 raft_consensus.cc:2258] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:26.898145 23384 tablet_replica.cc:330] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: stopping tablet replica
I20260809 22:43:26.898173 23360 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Bootstrap starting.
I20260809 22:43:26.898643 23384 raft_consensus.cc:2229] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:26.899206 23384 raft_consensus.cc:2258] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:26.899827 23356 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Bootstrap starting.
I20260809 22:43:26.900444 23360 tablet_bootstrap.cc:654] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.901490 23384 ts_tablet_manager.cc:1905] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:26.901575 23356 tablet_bootstrap.cc:654] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.905131 23357 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Bootstrap starting.
I20260809 22:43:26.906807 23357 tablet_bootstrap.cc:654] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:26.909569 23384 ts_tablet_manager.cc:1918] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:26.909776 23384 log.cc:1196] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/a65a7f4be0494b1f9fa09fe8ec94b930
I20260809 22:43:26.910406 23384 ts_tablet_manager.cc:1939] T a65a7f4be0494b1f9fa09fe8ec94b930 P 33cf6f19101a4839808ae15bc7887d89: Deleting consensus metadata
I20260809 22:43:26.912636 22805 catalog_manager.cc:4841] TS 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609): tablet a65a7f4be0494b1f9fa09fe8ec94b930 (table test-table [id=b401f96dd9b34758a9770263bf06b2b0]) successfully deleted
I20260809 22:43:26.916543 23356 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: No bootstrap required, opened a new log
I20260809 22:43:26.916698 23356 ts_tablet_manager.cc:1397] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.005s
I20260809 22:43:26.917460 23356 raft_consensus.cc:348] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.917690 23356 raft_consensus.cc:374] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.917768 23356 raft_consensus.cc:729] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7652638b9e34422da8d1d818ee196447, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.917996 23356 consensus_queue.cc:260] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.918092 23383 ts_tablet_manager.cc:1905] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:26.918519 23356 ts_tablet_manager.cc:1428] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:43:26.920071 23385 ts_tablet_manager.cc:1905] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:26.931108 23357 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: No bootstrap required, opened a new log
I20260809 22:43:26.931301 23357 ts_tablet_manager.cc:1397] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.006s
I20260809 22:43:26.932137 23357 raft_consensus.cc:348] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.932372 23357 raft_consensus.cc:374] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.932469 23357 raft_consensus.cc:729] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3282cdffca104c308570d41897afa394, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.932801 23357 consensus_queue.cc:260] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.935740 23357 ts_tablet_manager.cc:1428] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 22:43:26.937577 23383 ts_tablet_manager.cc:1918] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:26.939230 23385 ts_tablet_manager.cc:1918] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:26.939256 23360 tablet_bootstrap.cc:492] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: No bootstrap required, opened a new log
I20260809 22:43:26.939419 23360 ts_tablet_manager.cc:1397] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.007s
I20260809 22:43:26.939471 23383 log.cc:1196] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/a65a7f4be0494b1f9fa09fe8ec94b930
I20260809 22:43:26.940111 23383 ts_tablet_manager.cc:1939] T a65a7f4be0494b1f9fa09fe8ec94b930 P 3282cdffca104c308570d41897afa394: Deleting consensus metadata
I20260809 22:43:26.940279 23360 raft_consensus.cc:348] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.940511 23360 raft_consensus.cc:374] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:26.940613 23360 raft_consensus.cc:729] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33cf6f19101a4839808ae15bc7887d89, State: Initialized, Role: FOLLOWER
I20260809 22:43:26.940883 23385 log.cc:1196] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/a65a7f4be0494b1f9fa09fe8ec94b930
I20260809 22:43:26.941548 23385 ts_tablet_manager.cc:1939] T a65a7f4be0494b1f9fa09fe8ec94b930 P 7652638b9e34422da8d1d818ee196447: Deleting consensus metadata
I20260809 22:43:26.941510 23360 consensus_queue.cc:260] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:26.942200 23360 ts_tablet_manager.cc:1428] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:43:26.943579 22805 catalog_manager.cc:4841] TS 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297): tablet a65a7f4be0494b1f9fa09fe8ec94b930 (table test-table [id=b401f96dd9b34758a9770263bf06b2b0]) successfully deleted
I20260809 22:43:26.944967 22806 catalog_manager.cc:4841] TS 3282cdffca104c308570d41897afa394 (127.21.27.131:45011): tablet a65a7f4be0494b1f9fa09fe8ec94b930 (table test-table [id=b401f96dd9b34758a9770263bf06b2b0]) successfully deleted
I20260809 22:43:27.059577 23378 raft_consensus.cc:482] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:27.059804 23378 raft_consensus.cc:504] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.060839 23378 leader_election.cc:288] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609), 3282cdffca104c308570d41897afa394 (127.21.27.131:45011)
I20260809 22:43:27.061431 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf65e3322ca44603a9c37eb452f11d8d" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cf6f19101a4839808ae15bc7887d89" is_pre_election: true
I20260809 22:43:27.061417 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf65e3322ca44603a9c37eb452f11d8d" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394" is_pre_election: true
I20260809 22:43:27.061753 23290 raft_consensus.cc:2454] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 0.
I20260809 22:43:27.061753 22994 raft_consensus.cc:2454] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 0.
I20260809 22:43:27.062224 23061 leader_election.cc:302] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33cf6f19101a4839808ae15bc7887d89, 7652638b9e34422da8d1d818ee196447; no voters:
I20260809 22:43:27.062515 23378 raft_consensus.cc:2781] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:27.062661 23378 raft_consensus.cc:482] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:27.062798 23378 raft_consensus.cc:3037] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.065366 23378 raft_consensus.cc:504] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.065940 23378 leader_election.cc:288] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 election: Requested vote from peers 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609), 3282cdffca104c308570d41897afa394 (127.21.27.131:45011)
I20260809 22:43:27.066479 22994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf65e3322ca44603a9c37eb452f11d8d" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33cf6f19101a4839808ae15bc7887d89"
I20260809 22:43:27.066490 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf65e3322ca44603a9c37eb452f11d8d" candidate_uuid: "7652638b9e34422da8d1d818ee196447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394"
I20260809 22:43:27.066715 23290 raft_consensus.cc:3037] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.066715 22994 raft_consensus.cc:3037] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.069772 23290 raft_consensus.cc:2454] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 1.
I20260809 22:43:27.069770 22994 raft_consensus.cc:2454] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7652638b9e34422da8d1d818ee196447 in term 1.
I20260809 22:43:27.070295 23062 leader_election.cc:302] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3282cdffca104c308570d41897afa394, 7652638b9e34422da8d1d818ee196447; no voters:
I20260809 22:43:27.070691 23378 raft_consensus.cc:2781] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:27.070894 23378 raft_consensus.cc:686] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 1 LEADER]: Becoming Leader. State: Replica: 7652638b9e34422da8d1d818ee196447, State: Running, Role: LEADER
I20260809 22:43:27.071261 23378 consensus_queue.cc:237] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.074239 22823 catalog_manager.cc:5495] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7652638b9e34422da8d1d818ee196447 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "7652638b9e34422da8d1d818ee196447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } health_report { overall_health: UNKNOWN } } }
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315404, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:27.164238 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.145737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34895 (local address 127.0.0.1:34162)
0809 22:43:27.147757 (+ 2020us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:27.147780 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:27.147953 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:27.148427 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:27.148434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:27.149365 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:27.149794 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.149809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.151111 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.151118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.152671 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.152686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.153798 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.153808 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.153993 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.154738 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:27.154763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:27.158042 (+ 3279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:27.161393 (+ 3351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:27.161407 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:27.161414 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:27.161910 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:27.162383 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:27.162390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:27.162394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:27.162526 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:27.163073 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:27.163082 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:27.163305 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 22:43:27.163495 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 22:43:27.163804 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1828,"thread_start_us":1628,"threads_started":1}
I20260809 22:43:27.164300 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.145726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.21.27.190:34895)
0809 22:43:27.148130 (+ 2404us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.148134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.148154 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:43:27.148201 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.148205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.148260 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.148354 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.149966 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.150948 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.152829 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.153477 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.153624 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.158266 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.158271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.158275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.158303 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.161251 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.162060 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.162072 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.162074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.162263 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.162654 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.162661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.162665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.162941 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.163117 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.163604 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.163931 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":2280,"thread_start_us":2132,"threads_started":1}
I20260809 22:43:27.169736 22823 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:34162:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:43:27.170907 22823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:27.178092 22966 tablet_service.cc:1467] Processing CreateTablet for tablet be9b1b47e9a84f6098a52a85533cc008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89d552bfdb1e4cbe8e848f7ed98a8f35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:27.178162 23112 tablet_service.cc:1467] Processing CreateTablet for tablet be9b1b47e9a84f6098a52a85533cc008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89d552bfdb1e4cbe8e848f7ed98a8f35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:27.178673 22966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9b1b47e9a84f6098a52a85533cc008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:27.178762 23112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9b1b47e9a84f6098a52a85533cc008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:27.184331 23360 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: Bootstrap starting.
I20260809 22:43:27.185352 23265 tablet_service.cc:1467] Processing CreateTablet for tablet be9b1b47e9a84f6098a52a85533cc008 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=89d552bfdb1e4cbe8e848f7ed98a8f35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:27.185936 23360 tablet_bootstrap.cc:654] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:27.185957 23265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be9b1b47e9a84f6098a52a85533cc008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:27.187963 23360 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: No bootstrap required, opened a new log
I20260809 22:43:27.188154 23360 ts_tablet_manager.cc:1397] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 22:43:27.188926 23360 raft_consensus.cc:348] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.189191 23360 raft_consensus.cc:374] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:27.189280 23360 raft_consensus.cc:729] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33cf6f19101a4839808ae15bc7887d89, State: Initialized, Role: FOLLOWER
I20260809 22:43:27.189572 23360 consensus_queue.cc:260] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.189976 23356 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: Bootstrap starting.
I20260809 22:43:27.190199 23360 ts_tablet_manager.cc:1428] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 22:43:27.190666 23364 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.190968 23364 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.192144 23356 tablet_bootstrap.cc:654] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:27.196652 23357 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: Bootstrap starting.
I20260809 22:43:27.197288 23356 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: No bootstrap required, opened a new log
I20260809 22:43:27.197520 23356 ts_tablet_manager.cc:1397] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260809 22:43:27.198292 23356 raft_consensus.cc:348] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.198527 23357 tablet_bootstrap.cc:654] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:27.199505 23356 raft_consensus.cc:374] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:27.199687 23356 raft_consensus.cc:729] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7652638b9e34422da8d1d818ee196447, State: Initialized, Role: FOLLOWER
I20260809 22:43:27.199986 23356 consensus_queue.cc:260] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.200497 23378 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.200695 23356 ts_tablet_manager.cc:1428] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 22:43:27.200815 23378 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.201069 23357 tablet_bootstrap.cc:492] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: No bootstrap required, opened a new log
I20260809 22:43:27.201265 23357 ts_tablet_manager.cc:1397] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:43:27.202069 23357 raft_consensus.cc:348] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.202335 23357 raft_consensus.cc:374] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:27.202422 23357 raft_consensus.cc:729] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3282cdffca104c308570d41897afa394, State: Initialized, Role: FOLLOWER
I20260809 22:43:27.202665 23357 consensus_queue.cc:260] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.203159 23363 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.203501 23363 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.204136 23357 ts_tablet_manager.cc:1428] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 22:43:27.338146 23364 raft_consensus.cc:482] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:27.338342 23364 raft_consensus.cc:504] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.338975 23364 leader_election.cc:288] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297), 3282cdffca104c308570d41897afa394 (127.21.27.131:45011)
I20260809 22:43:27.339614 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9b1b47e9a84f6098a52a85533cc008" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7652638b9e34422da8d1d818ee196447" is_pre_election: true
I20260809 22:43:27.339663 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9b1b47e9a84f6098a52a85533cc008" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394" is_pre_election: true
I20260809 22:43:27.339936 23134 raft_consensus.cc:2454] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33cf6f19101a4839808ae15bc7887d89 in term 0.
I20260809 22:43:27.339986 23290 raft_consensus.cc:2454] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33cf6f19101a4839808ae15bc7887d89 in term 0.
I20260809 22:43:27.340416 22916 leader_election.cc:302] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33cf6f19101a4839808ae15bc7887d89, 7652638b9e34422da8d1d818ee196447; no voters:
I20260809 22:43:27.340705 23364 raft_consensus.cc:2781] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:27.340852 23364 raft_consensus.cc:482] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:27.340965 23364 raft_consensus.cc:3037] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.343061 23364 raft_consensus.cc:504] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.343740 23364 leader_election.cc:288] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 election: Requested vote from peers 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297), 3282cdffca104c308570d41897afa394 (127.21.27.131:45011)
I20260809 22:43:27.344221 23134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9b1b47e9a84f6098a52a85533cc008" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7652638b9e34422da8d1d818ee196447"
I20260809 22:43:27.344262 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be9b1b47e9a84f6098a52a85533cc008" candidate_uuid: "33cf6f19101a4839808ae15bc7887d89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3282cdffca104c308570d41897afa394"
I20260809 22:43:27.344450 23134 raft_consensus.cc:3037] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.344494 23290 raft_consensus.cc:3037] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:27.346967 23290 raft_consensus.cc:2454] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33cf6f19101a4839808ae15bc7887d89 in term 1.
I20260809 22:43:27.347242 23134 raft_consensus.cc:2454] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33cf6f19101a4839808ae15bc7887d89 in term 1.
I20260809 22:43:27.347481 22917 leader_election.cc:302] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3282cdffca104c308570d41897afa394, 33cf6f19101a4839808ae15bc7887d89; no voters:
I20260809 22:43:27.347810 23364 raft_consensus.cc:2781] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:27.348214 23364 raft_consensus.cc:686] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [term 1 LEADER]: Becoming Leader. State: Replica: 33cf6f19101a4839808ae15bc7887d89, State: Running, Role: LEADER
I20260809 22:43:27.348604 23364 consensus_queue.cc:237] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } }
I20260809 22:43:27.350538 23401 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: TxnStatusTablet state changed. Reason: New leader 33cf6f19101a4839808ae15bc7887d89. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.350822 23401 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:27.352273 23403 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:43:27.352196 22823 catalog_manager.cc:5495] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } health_report { overall_health: UNKNOWN } } }
I20260809 22:43:27.352586 23403 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:43:27.398471 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.391695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37609 (local address 127.0.0.1:48448)
0809 22:43:27.391949 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:27.391976 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:27.392147 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:27.392856 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:27.392860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:27.392871 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:27.393024 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.393032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.394645 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.394650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.395680 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.395692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.396682 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.396687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.396832 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.396896 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.397968 (+ 1072us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.397986 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 22:43:27.398065 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:43:27.398196 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:43:27.398602 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.391810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.21.27.129:37609)
0809 22:43:27.392356 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.392363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.392384 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:27.392478 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.392483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.392599 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.392791 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:27.393189 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.394496 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.395817 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.396512 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.396636 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.397094 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.397859 (+ 765us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.398072 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.398146 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.398232 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
I20260809 22:43:27.403558 23290 raft_consensus.cc:1261] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Refusing update from remote peer 33cf6f19101a4839808ae15bc7887d89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:27.403784 23134 raft_consensus.cc:1261] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Refusing update from remote peer 33cf6f19101a4839808ae15bc7887d89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:27.404284 23401 consensus_queue.cc:1035] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:27.404677 23364 consensus_queue.cc:1035] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:27.411725 23363 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: TxnStatusTablet state changed. Reason: New leader 33cf6f19101a4839808ae15bc7887d89. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.412015 23363 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.413936 23407 mvcc.cc:205] Tried to move back new op lower bound from 7316747908717481984 to 7316747908503453696. Current Snapshot: MvccSnapshot[applied={T|T < 7316747908717481984}]
I20260809 22:43:27.415120 23408 mvcc.cc:205] Tried to move back new op lower bound from 7316747908717481984 to 7316747908503453696. Current Snapshot: MvccSnapshot[applied={T|T < 7316747908717481984}]
I20260809 22:43:27.420178 23401 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.420502 23401 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:27.423019 23363 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.423394 23363 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 3282cdffca104c308570d41897afa394: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.428511 23378 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: TxnStatusTablet state changed. Reason: New leader 33cf6f19101a4839808ae15bc7887d89. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.428838 23378 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.431201 23401 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.431648 23401 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 33cf6f19101a4839808ae15bc7887d89: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:27.435202 23378 tablet_replica.cc:439] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33cf6f19101a4839808ae15bc7887d89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 } } peers { permanent_uuid: "7652638b9e34422da8d1d818ee196447" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40297 } } peers { permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 } } }
I20260809 22:43:27.435495 23378 tablet_replica.cc:442] T be9b1b47e9a84f6098a52a85533cc008 P 7652638b9e34422da8d1d818ee196447: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:27.448365 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.442167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37609 (local address 127.21.27.130:56107)
0809 22:43:27.442741 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:27.442756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:27.442869 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:27.443255 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:27.443258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:27.443268 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:27.443446 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.443453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.445078 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.445083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.446338 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.446350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.447329 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.447333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.447463 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.447517 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.447922 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.447930 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:43:27.447990 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 22:43:27.448102 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":260,"threads_started":1}
I20260809 22:43:27.448400 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.442373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:56107 (local address 127.21.27.129:37609)
0809 22:43:27.442562 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.442569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.442840 (+ 271us) server_negotiation.cc:407] Connection header received
0809 22:43:27.442987 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.442993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.443092 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.443218 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:27.443611 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.444906 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.446452 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.447182 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.447309 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.447692 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.447826 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.447917 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.448012 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.448098 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:43:27.459102 23290 raft_consensus.cc:1261] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Refusing update from remote peer 7652638b9e34422da8d1d818ee196447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:27.459221 22994 raft_consensus.cc:1261] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Refusing update from remote peer 7652638b9e34422da8d1d818ee196447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:27.459743 23369 consensus_queue.cc:1035] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3282cdffca104c308570d41897afa394" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:27.460214 23378 consensus_queue.cc:1035] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33cf6f19101a4839808ae15bc7887d89" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:27.501842 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.490656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40297 (local address 127.21.27.129:58369)
0809 22:43:27.491509 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:27.491535 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:27.492345 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:27.495884 (+ 3539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:27.495889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:27.495901 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:27.496361 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.496371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.497862 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.497867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.499359 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:27.499372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.500397 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.500403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:27.500560 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.500616 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.501026 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.501047 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 22:43:27.501158 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 22:43:27.501309 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":341,"threads_started":1}
I20260809 22:43:27.502513 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.490738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:58369 (local address 127.21.27.130:40297)
0809 22:43:27.495593 (+ 4855us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.495598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.495613 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:43:27.495664 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.495668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.495739 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.495854 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:27.496577 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.497734 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.499492 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.500163 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.500336 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.500832 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:27.500926 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:27.501023 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.502132 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.502218 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4740,"thread_start_us":4621,"threads_started":1}
I20260809 22:43:27.545114 22822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:27.545423 22822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:27.548403 22822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91e2f191227648b8a88933ab83e49be3: Sending DeleteTablet for 3 replicas of tablet bf65e3322ca44603a9c37eb452f11d8d
I20260809 22:43:27.549243 23112 tablet_service.cc:1514] Processing DeleteTablet for tablet bf65e3322ca44603a9c37eb452f11d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:27 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58466
I20260809 22:43:27.549393 23265 tablet_service.cc:1514] Processing DeleteTablet for tablet bf65e3322ca44603a9c37eb452f11d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:27 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:51656
I20260809 22:43:27.550066 22966 tablet_service.cc:1514] Processing DeleteTablet for tablet bf65e3322ca44603a9c37eb452f11d8d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:27 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43706
I20260809 22:43:27.569209 23428 tablet_replica.cc:330] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: stopping tablet replica
I20260809 22:43:27.569591 23428 raft_consensus.cc:2229] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:27.569932 23428 raft_consensus.cc:2258] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:27.571444 23428 ts_tablet_manager.cc:1905] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:27.571879 23427 tablet_replica.cc:330] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: stopping tablet replica
I20260809 22:43:27.572153 23427 raft_consensus.cc:2229] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:27.572678 23427 raft_consensus.cc:2258] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:27.573776 23427 ts_tablet_manager.cc:1905] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:27.574535 23429 tablet_replica.cc:330] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: stopping tablet replica
I20260809 22:43:27.574738 23429 raft_consensus.cc:2229] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:27.578560 23428 ts_tablet_manager.cc:1918] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:27.578730 23428 log.cc:1196] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/bf65e3322ca44603a9c37eb452f11d8d
I20260809 22:43:27.579296 23428 ts_tablet_manager.cc:1939] T bf65e3322ca44603a9c37eb452f11d8d P 3282cdffca104c308570d41897afa394: Deleting consensus metadata
I20260809 22:43:27.579387 23429 raft_consensus.cc:2258] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:27.580408 22806 catalog_manager.cc:4841] TS 3282cdffca104c308570d41897afa394 (127.21.27.131:45011): tablet bf65e3322ca44603a9c37eb452f11d8d (table test-table [id=7633dca43a794b7aaf7bf1548ab6d141]) successfully deleted
I20260809 22:43:27.580597 23429 ts_tablet_manager.cc:1905] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:27.589835 23427 ts_tablet_manager.cc:1918] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:27.590088 23427 log.cc:1196] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/bf65e3322ca44603a9c37eb452f11d8d
I20260809 22:43:27.590688 23427 ts_tablet_manager.cc:1939] T bf65e3322ca44603a9c37eb452f11d8d P 7652638b9e34422da8d1d818ee196447: Deleting consensus metadata
I20260809 22:43:27.592074 22805 catalog_manager.cc:4841] TS 7652638b9e34422da8d1d818ee196447 (127.21.27.130:40297): tablet bf65e3322ca44603a9c37eb452f11d8d (table test-table [id=7633dca43a794b7aaf7bf1548ab6d141]) successfully deleted
I20260809 22:43:27.596215 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.561224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.21.27.129:37609)
0809 22:43:27.567410 (+ 6186us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.567422 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.574465 (+ 7043us) server_negotiation.cc:407] Connection header received
0809 22:43:27.574779 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.574785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.574869 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.579260 (+ 4391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.579486 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.580574 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.582097 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.582782 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.582946 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.588415 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.588430 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.588449 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.588515 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.592868 (+ 4353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.593732 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.593742 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.593746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.593969 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.594361 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.594366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.594369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.594866 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.595251 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.595561 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.595783 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6013}
I20260809 22:43:27.596414 23429 ts_tablet_manager.cc:1918] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:27.596593 23429 log.cc:1196] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/bf65e3322ca44603a9c37eb452f11d8d
I20260809 22:43:27.597149 23429 ts_tablet_manager.cc:1939] T bf65e3322ca44603a9c37eb452f11d8d P 33cf6f19101a4839808ae15bc7887d89: Deleting consensus metadata
W20260809 22:43:27.597146 23036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48462
I20260809 22:43:27.599536 22805 catalog_manager.cc:4841] TS 33cf6f19101a4839808ae15bc7887d89 (127.21.27.129:37609): tablet bf65e3322ca44603a9c37eb452f11d8d (table test-table [id=7633dca43a794b7aaf7bf1548ab6d141]) successfully deleted
I20260809 22:43:27.624289 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.604353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.21.27.190:34895)
0809 22:43:27.604627 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.604636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.610838 (+ 6202us) server_negotiation.cc:407] Connection header received
0809 22:43:27.611045 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.611051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.611191 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.611304 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.612663 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.613489 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.614834 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.615349 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.615491 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.618103 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.618108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.618112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.618146 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.621209 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.621876 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.621883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.621887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.622074 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.622475 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.622481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.622486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.622851 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.623301 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.623578 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.623886 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260809 22:43:27.625280 22822 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41568
I20260809 22:43:27.628357 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22900
I20260809 22:43:27.649796 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.629164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.21.27.129:37609)
0809 22:43:27.629376 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.629385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.629865 (+ 480us) server_negotiation.cc:407] Connection header received
0809 22:43:27.631309 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.631315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.631701 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.631881 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.634170 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.635338 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.637663 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.638340 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.638443 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.642325 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.642331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.642334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.642382 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.646028 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.647254 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.647258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.647261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.647417 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.647942 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.647946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.647948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.648354 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.648692 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.649086 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.649325 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 22:43:27.650462 23036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48472
W20260809 22:43:27.651088 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22900: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:27.651404 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22900
I20260809 22:43:27.675128 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23045
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:27.691025 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.675950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59628 (local address 127.21.27.130:40297)
0809 22:43:27.676182 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.676189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.676212 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:43:27.676419 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.676431 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.676504 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.676614 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.677927 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.678671 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.680136 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.680534 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.680669 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.685047 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.685069 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.685085 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.685122 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.688146 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.688857 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.688871 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.688873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.689011 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.689426 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.689432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.689435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.689805 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.690163 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.690422 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.690654 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:43:27.691910 23186 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59628
W20260809 22:43:27.692634 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23045: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:27.692822 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23045
I20260809 22:43:27.716944 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23195
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[22774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:27.736066 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:27.717774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.21.27.131:45011)
0809 22:43:27.718288 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:27.718292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:27.718309 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:43:27.718365 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:27.718369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:27.718435 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:27.718527 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:27.719974 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.720675 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.722226 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:27.722659 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:27.722755 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:27.726819 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:27.726833 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:27.726850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:27.726883 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:27.731082 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.732256 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.732336 (+ 80us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.732342 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.732639 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:27.733235 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:27.733243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:27.733247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:27.733821 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:27.735070 (+ 1249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:27.735439 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 22:43:27.735684 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":302,"threads_started":1}
W20260809 22:43:27.738209 23331 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55962
W20260809 22:43:27.739082 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23195: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:27.739301 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23195
I20260809 22:43:27.759012 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 22787
I20260809 22:43:27.760301 22868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:27.875908 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 22787
2026-08-09T22:43:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4730 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[23440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[23440](info): set up 2 sockets
Aug 09 22:43:27 dist-test-slave-chsk krb5kdc[23440](info): commencing operation
krb5kdc: starting...
W20260809 22:43:29.012609 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:29 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:29.226635 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:36521
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:36521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:29.545118 23454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:29.545372 23454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:29.545466 23454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:29.552506 23454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:29.552597 23454 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:43:29.552639 23454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:29.552697 23454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:29.552748 23454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:29.552803 23454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:29.562109 23454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:36521
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:36521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:29.563310 23454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:29.564590 23454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:29.575768 23468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:29.575840 23465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:29.575858 23466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:29.577147 23454 server_base.cc:1034] running on GCE node
I20260809 22:43:29.577797 23454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:29.578971 23454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:29.580181 23454 hybrid_clock.cc:648] HybridClock initialized: now 1786315409580093 us; error 56 us; skew 500 ppm
I20260809 22:43:29.580451 23454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:29 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315409, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:29.583528 23454 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:29.584945 23454 webserver.cc:460] Webserver started at http://127.21.27.190:45657/ using document root <none> and password file <none>
I20260809 22:43:29.585726 23454 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:29.585840 23454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:29.586252 23454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:29.589269 23454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "a9ec85978630467f8727e5201d63d9ba"
format_stamp: "Formatted at 2026-08-09 22:43:29 on dist-test-slave-chsk"
server_key: "d4bda03b910d964167849c7950aedeaa"
server_key_iv: "ee65357598f7edf3e086355d2014f72d"
server_key_version: "encryptionkey@0"
I20260809 22:43:29.590058 23454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "a9ec85978630467f8727e5201d63d9ba"
format_stamp: "Formatted at 2026-08-09 22:43:29 on dist-test-slave-chsk"
server_key: "d4bda03b910d964167849c7950aedeaa"
server_key_iv: "ee65357598f7edf3e086355d2014f72d"
server_key_version: "encryptionkey@0"
I20260809 22:43:29.595324 23454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:43:29.598917 23476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:29.600250 23454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:29.600414 23454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "a9ec85978630467f8727e5201d63d9ba"
format_stamp: "Formatted at 2026-08-09 22:43:29 on dist-test-slave-chsk"
server_key: "d4bda03b910d964167849c7950aedeaa"
server_key_iv: "ee65357598f7edf3e086355d2014f72d"
server_key_version: "encryptionkey@0"
I20260809 22:43:29.600638 23454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:29.618150 23454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:29.622319 23454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:29.622622 23454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:29.644737 23454 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:36521
I20260809 22:43:29.644758 23542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:36521 every 8 connection(s)
I20260809 22:43:29.646126 23454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:29.649791 23543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:29.655651 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23454
I20260809 22:43:29.655869 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:29.656551 21614 external_mini_cluster.cc:1428] Setting key fe978a11bb27bc6b4daeb6537a84f480
I20260809 22:43:29.661177 23543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Bootstrap starting.
I20260809 22:43:29.665102 23543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:29.666394 23543 log.cc:824] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:29.669827 23543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: No bootstrap required, opened a new log
Aug 09 22:43:29 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:29.679574 23543 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } }
I20260809 22:43:29.680058 23543 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:29.680166 23543 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ec85978630467f8727e5201d63d9ba, State: Initialized, Role: FOLLOWER
I20260809 22:43:29.680848 23543 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } }
I20260809 22:43:29.681172 23543 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:29.681311 23543 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:29.681496 23543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:29.681914 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:29.658668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.21.27.190:36521)
0809 22:43:29.659523 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:29.659539 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:29.659593 (+ 54us) server_negotiation.cc:407] Connection header received
0809 22:43:29.661945 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:29.661971 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:29.662459 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:29.662901 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:29.664275 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:29.666287 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:29.668070 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:29.669088 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:29.669275 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:29.673284 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:29.673294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:29.673313 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:29.673354 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:29.677368 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:29.678059 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:29.678069 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:29.678079 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:29.678297 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:29.678689 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:29.678694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:29.678698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:29.679414 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:29.679744 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:29.680047 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:43:29.680495 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":397,"threads_started":1}
I20260809 22:43:29.684146 23543 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } }
I20260809 22:43:29.684648 23543 leader_election.cc:302] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9ec85978630467f8727e5201d63d9ba; no voters:
I20260809 22:43:29.685214 23543 leader_election.cc:288] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:29.685361 23548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:29.686116 23548 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [term 1 LEADER]: Becoming Leader. State: Replica: a9ec85978630467f8727e5201d63d9ba, State: Running, Role: LEADER
I20260809 22:43:29.686751 23548 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } }
I20260809 22:43:29.687343 23543 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:29.689934 23550 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9ec85978630467f8727e5201d63d9ba. Latest consensus state: current_term: 1 leader_uuid: "a9ec85978630467f8727e5201d63d9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } } }
I20260809 22:43:29.690344 23550 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:29.690971 23549 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9ec85978630467f8727e5201d63d9ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec85978630467f8727e5201d63d9ba" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36521 } } }
I20260809 22:43:29.691368 23549 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba [sys.catalog]: This master's current role is: LEADER
W20260809 22:43:29.701265 23566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:43:29.701419 23566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:43:29.701575 23556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:29.703225 23556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:29.711477 23556 catalog_manager.cc:1347] Generated new cluster ID: 4d0e1da71e824a459d572a8233f4de73
I20260809 22:43:29.711601 23556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:29.720607 23556 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:29.722512 23556 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:29.734591 23556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Generated new TSK 0
I20260809 22:43:29.735688 23556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:29.794236 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:36521
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:30.122148 23571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:30.122457 23571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:30.122658 23571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:30.130282 23571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:30.130422 23571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:30.130595 23571 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:43:30.130681 23571 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:30.130775 23571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:30.141153 23571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:30.142593 23571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:30.143968 23571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:30.157209 23576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:30.157120 23579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:30.157150 23577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:30.158300 23571 server_base.cc:1034] running on GCE node
I20260809 22:43:30.159005 23571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:30.160032 23571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:30.161293 23571 hybrid_clock.cc:648] HybridClock initialized: now 1786315410161237 us; error 37 us; skew 500 ppm
I20260809 22:43:30.161676 23571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:30 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:30.166078 23571 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:30.167699 23571 webserver.cc:460] Webserver started at http://127.21.27.129:36303/ using document root <none> and password file <none>
I20260809 22:43:30.168442 23571 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:30.168602 23571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:30.168929 23571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:30.172075 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "ee37ae34072e4058b9e9958a0a1e8682"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "cd6ad0978dda53029c793d005b568621"
server_key_iv: "715adea23a06c66832789ad5c0491d66"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.172904 23571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "ee37ae34072e4058b9e9958a0a1e8682"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "cd6ad0978dda53029c793d005b568621"
server_key_iv: "715adea23a06c66832789ad5c0491d66"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.178529 23571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:30.182374 23588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:30.183960 23571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:30.184175 23571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "ee37ae34072e4058b9e9958a0a1e8682"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "cd6ad0978dda53029c793d005b568621"
server_key_iv: "715adea23a06c66832789ad5c0491d66"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.184396 23571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:30.202036 23571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:30.206552 23571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:30.206871 23571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:30.209599 23571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:30.209743 23571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:30.209877 23571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:30.209967 23571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:43:30 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:30.236967 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.210328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50519 (local address 127.21.27.190:36521)
0809 22:43:30.212574 (+ 2246us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:30.212583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:30.212608 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:43:30.214160 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:30.214176 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:30.214273 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:30.214375 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:30.216484 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.217804 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.220460 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.221199 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.221366 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.227449 (+ 6083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:30.227460 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:30.227467 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:30.227515 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:30.231330 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.232358 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.232367 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.232373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.232564 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.233366 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.233370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.233372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.233815 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:30.233997 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:30.236476 (+ 2479us) server_negotiation.cc:299] Negotiation successful
0809 22:43:30.236678 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2105,"thread_start_us":1982,"threads_started":1}
I20260809 22:43:30.238059 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.210895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.129:50519)
0809 22:43:30.211806 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:30.211852 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:30.213862 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:30.214608 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:30.214619 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:30.215439 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:30.216262 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.216297 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.217989 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.217996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.220243 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.220258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.221479 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.221488 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.221853 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.222810 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:30.222836 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:30.226971 (+ 4135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:30.231518 (+ 4547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.231533 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.231559 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.232199 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.232737 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.232746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.232753 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.233192 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.234103 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:30.234114 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:30.234737 (+ 623us) client_negotiation.cc:769] Sending connection context
0809 22:43:30.235065 (+ 328us) client_negotiation.cc:240] Negotiation successful
0809 22:43:30.235655 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":407,"threads_started":1}
I20260809 22:43:30.255066 23571 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:45965
I20260809 22:43:30.255133 23717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:45965 every 8 connection(s)
I20260809 22:43:30.256490 23571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:30.258984 23718 heartbeater.cc:347] Connected to a master server at 127.21.27.190:36521
I20260809 22:43:30.259299 23718 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:30.260165 23718 heartbeater.cc:511] Master 127.21.27.190:36521 requested a full tablet report, sending...
I20260809 22:43:30.262564 23493 ts_manager.cc:194] Registered new tserver with Master: ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965)
I20260809 22:43:30.263909 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.247437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:51075 (local address 127.21.27.190:36521)
0809 22:43:30.247661 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:30.247669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:30.248543 (+ 874us) server_negotiation.cc:407] Connection header received
0809 22:43:30.248809 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:30.248818 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:30.248920 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:30.249046 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:30.250701 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.252076 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.254435 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.255016 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.255193 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.257431 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:30.257437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:30.257442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:30.257479 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:30.261054 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.261885 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.261889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.261891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.262064 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.262509 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.262513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.262516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.262890 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:30.263010 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:30.263495 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 22:43:30.263671 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:43:30.264176 23697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.247387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.129:51075)
0809 22:43:30.248374 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:30.248409 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:30.248620 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:30.249132 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:30.249139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:30.249928 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:30.250500 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.250517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.252312 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.252319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.254221 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.254240 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.255358 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.255367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.255501 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.256184 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:30.256202 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:30.257179 (+ 977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:30.261217 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.261221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.261224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.261767 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.262196 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.262203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.262208 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.262367 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.263015 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:30.263020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:30.263247 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 22:43:30.263415 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 22:43:30.263738 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":601,"threads_started":1}
I20260809 22:43:30.264598 23493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:50519
I20260809 22:43:30.265601 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23571
I20260809 22:43:30.265849 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:30.266705 21614 external_mini_cluster.cc:1428] Setting key e740fabda7f07928b653172a717cac0b
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:30.310707 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:36521
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:30.638526 23722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:30.638835 23722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:30.638943 23722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:30.646615 23722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:30.646736 23722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:30.646879 23722 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:43:30.646940 23722 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:30.647109 23722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:30.657002 23722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:30.658262 23722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:30.659627 23722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:30.672860 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:30.672981 23729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:30.673014 23727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:30.673745 23722 server_base.cc:1034] running on GCE node
I20260809 22:43:30.674387 23722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:30.675369 23722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:30.676589 23722 hybrid_clock.cc:648] HybridClock initialized: now 1786315410676548 us; error 54 us; skew 500 ppm
I20260809 22:43:30.676936 23722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:30 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:30.680132 23722 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:30.681806 23722 webserver.cc:460] Webserver started at http://127.21.27.130:45365/ using document root <none> and password file <none>
I20260809 22:43:30.682531 23722 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:30.682678 23722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:30.682997 23722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:30.685930 23722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "3549fdf0dd9340ee948a67acd8bae650"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "1d53826b5d3680c39d801852e99aa84c"
server_key_iv: "a99250952312b17f4a939070eb41fefd"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.686694 23722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "3549fdf0dd9340ee948a67acd8bae650"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "1d53826b5d3680c39d801852e99aa84c"
server_key_iv: "a99250952312b17f4a939070eb41fefd"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.692366 23722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 22:43:30.696025 23739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:30.697388 23722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:30.697580 23722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "3549fdf0dd9340ee948a67acd8bae650"
format_stamp: "Formatted at 2026-08-09 22:43:30 on dist-test-slave-chsk"
server_key: "1d53826b5d3680c39d801852e99aa84c"
server_key_iv: "a99250952312b17f4a939070eb41fefd"
server_key_version: "encryptionkey@0"
I20260809 22:43:30.697788 23722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:30.713485 23722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:30.717538 23722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:30.717846 23722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:30.720660 23722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:30.720817 23722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:30.720975 23722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:30.721077 23722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:43:30 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315410, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:30.745525 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.721266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:48103 (local address 127.21.27.190:36521)
0809 22:43:30.721470 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:30.721477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:30.722845 (+ 1368us) server_negotiation.cc:407] Connection header received
0809 22:43:30.724918 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:30.724927 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:30.725035 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:30.725179 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:30.727598 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.728771 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.731322 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.732107 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.732283 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.737188 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:30.737197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:30.737205 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:30.737262 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:30.741320 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.742564 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.742577 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.742581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.742742 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.743436 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.743441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.743445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.743733 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:30.743863 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:30.745076 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0809 22:43:30.745286 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 22:43:30.748023 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.721722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.130:48103)
0809 22:43:30.722640 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:30.722697 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:30.724597 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:30.725476 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:30.725486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:30.726315 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:30.727309 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.727345 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.728923 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.728928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.731097 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.731135 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.732353 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.732358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.732531 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.733323 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:30.733359 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:30.736890 (+ 3531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:30.741540 (+ 4650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.741555 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.741573 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.742418 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.742892 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.742901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.742905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.743280 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.743873 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:30.743885 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:30.744498 (+ 613us) client_negotiation.cc:769] Sending connection context
0809 22:43:30.744919 (+ 421us) client_negotiation.cc:240] Negotiation successful
0809 22:43:30.745980 (+ 1061us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":413,"threads_started":1}
I20260809 22:43:30.770380 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.753781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.130:42817)
0809 22:43:30.754536 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:30.754571 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:30.754743 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:30.755173 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:30.755180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:30.756064 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:30.756537 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.756555 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.757807 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.757814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.759436 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:30.759463 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.760605 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.760612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:30.760764 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.761394 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:30.761411 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:30.762157 (+ 746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:30.766718 (+ 4561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.766724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.766729 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.767512 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.768064 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:30.768069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:30.768073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:30.768216 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:30.768827 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:30.768832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:30.769500 (+ 668us) client_negotiation.cc:769] Sending connection context
0809 22:43:30.769719 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 22:43:30.770056 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":379,"threads_started":1}
I20260809 22:43:30.770714 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:30.753766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:42817 (local address 127.21.27.190:36521)
0809 22:43:30.753964 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:30.753972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:30.754673 (+ 701us) server_negotiation.cc:407] Connection header received
0809 22:43:30.754919 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:30.754924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:30.754986 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:30.755084 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:30.756726 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.757659 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.759616 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:30.760395 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:30.760554 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:30.762363 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:30.762372 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:30.762380 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:30.762443 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:30.766582 (+ 4139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.767722 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.767730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.767736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.767928 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:30.768365 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:30.768372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:30.768375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:30.768701 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:30.768864 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:30.769891 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0809 22:43:30.770212 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:43:30.780411 23722 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:38455
I20260809 22:43:30.780479 23868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:38455 every 8 connection(s)
I20260809 22:43:30.781643 23722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:30.783643 23869 heartbeater.cc:347] Connected to a master server at 127.21.27.190:36521
I20260809 22:43:30.783869 23869 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:30.784487 23869 heartbeater.cc:511] Master 127.21.27.190:36521 requested a full tablet report, sending...
I20260809 22:43:30.785797 23492 ts_manager.cc:194] Registered new tserver with Master: 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455)
I20260809 22:43:30.786731 23492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:48103
I20260809 22:43:30.791460 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23722
I20260809 22:43:30.791639 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:30.792055 21614 external_mini_cluster.cc:1428] Setting key 3779a841771caae9b7aa3278c3b08266
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:30.838500 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:36521
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:31.161399 23873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:31.161660 23873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:31.161758 23873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:31.169798 23873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:31.169903 23873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:31.169989 23873 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:43:31.170045 23873 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:43:31.170117 23873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:31.179597 23873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:31.180768 23873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:31.181978 23873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:31.195387 23879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:31.195526 23878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:31.195451 23881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:31.196122 23873 server_base.cc:1034] running on GCE node
I20260809 22:43:31.196717 23873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:31.197551 23873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:31.198743 23873 hybrid_clock.cc:648] HybridClock initialized: now 1786315411198703 us; error 29 us; skew 500 ppm
I20260809 22:43:31.199052 23873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:31 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315411, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:31.205559 23873 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:31.206895 23873 webserver.cc:460] Webserver started at http://127.21.27.131:37457/ using document root <none> and password file <none>
I20260809 22:43:31.207677 23873 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:31.207823 23873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:31.208134 23873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:31.210948 23873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "69725a2e618f4dfe9c99c983eb1e1939"
format_stamp: "Formatted at 2026-08-09 22:43:31 on dist-test-slave-chsk"
server_key: "24aad3774f403489db97fce208486828"
server_key_iv: "977a4af25d96db0caf0c0b200c7f94c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:31.211758 23873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "69725a2e618f4dfe9c99c983eb1e1939"
format_stamp: "Formatted at 2026-08-09 22:43:31 on dist-test-slave-chsk"
server_key: "24aad3774f403489db97fce208486828"
server_key_iv: "977a4af25d96db0caf0c0b200c7f94c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:31.216579 23873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:43:31.220466 23889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:31.221755 23873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:31.221933 23873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "69725a2e618f4dfe9c99c983eb1e1939"
format_stamp: "Formatted at 2026-08-09 22:43:31 on dist-test-slave-chsk"
server_key: "24aad3774f403489db97fce208486828"
server_key_iv: "977a4af25d96db0caf0c0b200c7f94c6"
server_key_version: "encryptionkey@0"
I20260809 22:43:31.222133 23873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:31.259171 23873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:31.263653 23873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:31.263892 23873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:31.266340 23873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:31.266489 23873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:31.266631 23873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:31.266726 23873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:31.269306 23718 heartbeater.cc:503] Master 127.21.27.190:36521 was elected leader, sending a full tablet report...
Aug 09 22:43:31 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315411, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:31.291241 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.267173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:41143 (local address 127.21.27.190:36521)
0809 22:43:31.267373 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.267379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.268636 (+ 1257us) server_negotiation.cc:407] Connection header received
0809 22:43:31.271248 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.271253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.271339 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.271452 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:31.273988 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.274908 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.277465 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.278195 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.278368 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.283554 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:31.283567 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:31.283572 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:31.283604 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:31.286676 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.287803 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.287811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.287815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.287985 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.288671 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.288678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.288684 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.289198 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:31.289381 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.290463 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.290746 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:43:31.293116 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.267649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.131:41143)
0809 22:43:31.268464 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.268513 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.270938 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.271730 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.271752 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.272708 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:31.273777 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.273811 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.275192 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.275201 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.277237 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.277253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.278468 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.278499 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.278731 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.279527 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:31.279551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:31.283319 (+ 3768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:31.286837 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.286845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.286891 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.287601 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.288128 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.288136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.288141 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.288506 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.289346 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:31.289358 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:31.289899 (+ 541us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.290304 (+ 405us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.290993 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":375,"threads_started":1}
I20260809 22:43:31.313944 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.299194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:35053 (local address 127.21.27.190:36521)
0809 22:43:31.299423 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.299431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.300433 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 22:43:31.300696 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.300702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.300802 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.300898 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:31.302547 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.303412 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.305093 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.305521 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.305678 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.307586 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:31.307595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:31.307602 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:31.307648 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:31.310940 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.311902 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.311909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.311915 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.312078 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.312436 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.312441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.312444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.312780 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:31.312939 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.313349 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.313586 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:43:31.314038 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.299381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.21.27.131:35053)
0809 22:43:31.300296 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.300332 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.300505 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.301059 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.301065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.301819 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:31.302395 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.302409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.303551 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.303557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.304943 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.304955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.305720 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.305726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.305893 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.306585 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:31.306601 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:31.307394 (+ 793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:31.311074 (+ 3680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.311079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.311087 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.311716 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.312176 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.312182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.312186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.312314 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.312891 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:31.312896 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:31.313096 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.313233 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.313564 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":540,"threads_started":1}
I20260809 22:43:31.317343 23873 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:36763
I20260809 22:43:31.317441 24020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:36763 every 8 connection(s)
I20260809 22:43:31.318867 23873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:31.319948 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 23873
I20260809 22:43:31.320132 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:31.320753 21614 external_mini_cluster.cc:1428] Setting key 0e80f95d656a1ea3f1bdd6c822624202
I20260809 22:43:31.320981 24021 heartbeater.cc:347] Connected to a master server at 127.21.27.190:36521
I20260809 22:43:31.321691 24021 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:31.322444 24021 heartbeater.cc:511] Master 127.21.27.190:36521 requested a full tablet report, sending...
I20260809 22:43:31.323834 23493 ts_manager.cc:194] Registered new tserver with Master: 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:31.324589 23493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:41143
I20260809 22:43:31.335928 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:43:31 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:43:31 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:31.388572 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.374497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.21.27.190:36521)
0809 22:43:31.374766 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.374772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.375121 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:43:31.375396 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.375401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.375467 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.375577 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:31.377033 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.377790 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.379032 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.379525 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.379662 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.383654 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:31.383659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:31.383662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:31.383702 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:31.386084 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.386651 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.386654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.386657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.386777 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.387181 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.387189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.387195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.387485 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:31.387659 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.387932 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.388186 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:43:31.392880 23493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:31.399919 23493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:31.421962 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.411816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:38455 (local address 127.0.0.1:36850)
0809 22:43:31.413072 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.413103 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.413308 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.414055 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.414067 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.414111 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.414574 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.414588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.416645 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.416651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.419098 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.419113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.420924 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.420930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.421124 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.421178 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.421320 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.421512 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":1,"thread_start_us":939,"threads_started":1}
I20260809 22:43:31.423035 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.412094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45965 (local address 127.0.0.1:43966)
0809 22:43:31.414330 (+ 2236us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.414348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.414476 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.415377 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.415382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.415406 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.415818 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.415830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.417658 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.417663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.419803 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.419816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.421838 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.421845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.422376 (+ 531us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.422419 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.422500 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.422622 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2107,"mutex_wait_us":1,"thread_start_us":2364,"threads_started":1}
I20260809 22:43:31.423318 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.412017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.21.27.130:38455)
0809 22:43:31.412632 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.412642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.413218 (+ 576us) server_negotiation.cc:407] Connection header received
0809 22:43:31.413525 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.413535 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.413791 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.414804 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.415010 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.416495 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.419342 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.420687 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.422507 (+ 1820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.422705 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.422794 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.422963 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":412,"threads_started":1}
I20260809 22:43:31.424754 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.412414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.21.27.129:45965)
0809 22:43:31.413863 (+ 1449us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.413873 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.414621 (+ 748us) server_negotiation.cc:407] Connection header received
0809 22:43:31.414733 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.414740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.414939 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.415122 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.416006 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.417491 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.419975 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.421367 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.421491 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.421689 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.424314 (+ 2625us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.424482 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":1726,"threads_started":1}
I20260809 22:43:31.427062 23792 tablet_service.cc:1467] Processing CreateTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 (DEFAULT_TABLE table=test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.428447 23792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98727ebe4f9f4893b9e3e307e42a8670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.428071 23644 tablet_service.cc:1467] Processing CreateTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 (DEFAULT_TABLE table=test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.429333 23644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98727ebe4f9f4893b9e3e307e42a8670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.432435 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.414030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:36763 (local address 127.0.0.1:43914)
0809 22:43:31.422237 (+ 8207us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.422254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.422343 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.425588 (+ 3245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.425592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.425617 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.425845 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.425860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.427905 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.427910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.429799 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.429816 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.431678 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.431685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.431889 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.431935 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.432006 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.432146 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8115,"thread_start_us":10220,"threads_started":1}
I20260809 22:43:31.433612 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.414733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43914 (local address 127.21.27.131:36763)
0809 22:43:31.424966 (+ 10233us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.424975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.425008 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:43:31.425123 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.425130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.425358 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.425596 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.426050 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.427677 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.429999 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.431482 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.432788 (+ 1306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.433028 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.433099 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.433273 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10027,"thread_start_us":10459,"threads_started":1}
I20260809 22:43:31.437206 23941 tablet_service.cc:1467] Processing CreateTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 (DEFAULT_TABLE table=test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.438647 23941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98727ebe4f9f4893b9e3e307e42a8670. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.440786 24042 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Bootstrap starting.
I20260809 22:43:31.442692 24043 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Bootstrap starting.
I20260809 22:43:31.445567 24043 tablet_bootstrap.cc:654] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.445921 24042 tablet_bootstrap.cc:654] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.446684 24043 log.cc:824] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:31.447110 24042 log.cc:824] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:31.449927 24042 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: No bootstrap required, opened a new log
I20260809 22:43:31.449986 24043 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: No bootstrap required, opened a new log
I20260809 22:43:31.450241 24042 ts_tablet_manager.cc:1397] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 22:43:31.450273 24043 ts_tablet_manager.cc:1397] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260809 22:43:31.453948 24046 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Bootstrap starting.
I20260809 22:43:31.457769 24046 tablet_bootstrap.cc:654] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.458655 24043 raft_consensus.cc:348] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.459043 24043 raft_consensus.cc:374] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.459110 24046 log.cc:824] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:31.459218 24043 raft_consensus.cc:729] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.459928 24043 consensus_queue.cc:260] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.460844 24042 raft_consensus.cc:348] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.461303 24042 raft_consensus.cc:374] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.461405 24042 raft_consensus.cc:729] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3549fdf0dd9340ee948a67acd8bae650, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.461805 24043 ts_tablet_manager.cc:1428] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Time spent starting tablet: real 0.011s user 0.007s sys 0.006s
I20260809 22:43:31.461958 24042 consensus_queue.cc:260] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.462468 24046 tablet_bootstrap.cc:492] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: No bootstrap required, opened a new log
I20260809 22:43:31.462755 24046 ts_tablet_manager.cc:1397] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:43:31.465984 23869 heartbeater.cc:503] Master 127.21.27.190:36521 was elected leader, sending a full tablet report...
I20260809 22:43:31.466681 24042 ts_tablet_manager.cc:1428] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260809 22:43:31.472287 24046 raft_consensus.cc:348] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.472640 24046 raft_consensus.cc:374] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.472710 24046 raft_consensus.cc:729] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.473173 24046 consensus_queue.cc:260] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.474097 24021 heartbeater.cc:503] Master 127.21.27.190:36521 was elected leader, sending a full tablet report...
I20260809 22:43:31.474563 24046 ts_tablet_manager.cc:1428] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 22:43:31.499528 24050 raft_consensus.cc:482] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:31.499972 24050 raft_consensus.cc:504] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.501840 24050 leader_election.cc:288] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965)
I20260809 22:43:31.507637 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.502438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:39883 (local address 127.21.27.129:45965)
0809 22:43:31.502658 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.502676 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.502697 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:31.502781 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.502788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.502897 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.502999 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.503510 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.504304 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.505963 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.506710 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.506876 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.506957 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.507268 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.507394 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:43:31.507637 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.502107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45965 (local address 127.21.27.131:39883)
0809 22:43:31.502303 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.502325 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.502450 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.503057 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.503066 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.503097 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.503359 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.503371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.504447 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.504452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.505799 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.505810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.506863 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.506869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.507062 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.507085 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.507235 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.507363 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260809 22:43:31.508540 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.502282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:33305 (local address 127.21.27.130:38455)
0809 22:43:31.502494 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.502500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.503382 (+ 882us) server_negotiation.cc:407] Connection header received
0809 22:43:31.503586 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.503598 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.503659 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.503759 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.504257 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.505139 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.506774 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.507616 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.507798 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.507869 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.508156 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.508277 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:43:31.508607 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.502107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:38455 (local address 127.21.27.131:33305)
0809 22:43:31.503264 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.503286 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.503411 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.503788 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.503792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.503817 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.504116 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.504125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.505272 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.505277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.506624 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.506635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.507814 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.507820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.507995 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.508032 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.508120 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.508236 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":436,"thread_start_us":290,"threads_started":1}
W20260809 22:43:31.509029 23719 tablet.cc:2369] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:31.508991 23666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98727ebe4f9f4893b9e3e307e42a8670" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee37ae34072e4058b9e9958a0a1e8682" is_pre_election: true
I20260809 22:43:31.509550 23666 raft_consensus.cc:2454] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 0.
I20260809 22:43:31.510468 23890 leader_election.cc:302] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:31.510838 24050 raft_consensus.cc:2781] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:31.510994 24050 raft_consensus.cc:482] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:31.511108 24050 raft_consensus.cc:3037] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.510993 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98727ebe4f9f4893b9e3e307e42a8670" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650" is_pre_election: true
I20260809 22:43:31.511618 23821 raft_consensus.cc:2454] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 0.
I20260809 22:43:31.513810 24050 raft_consensus.cc:504] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.514386 24050 leader_election.cc:288] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 election: Requested vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965)
I20260809 22:43:31.514775 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98727ebe4f9f4893b9e3e307e42a8670" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650"
I20260809 22:43:31.514863 23666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98727ebe4f9f4893b9e3e307e42a8670" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee37ae34072e4058b9e9958a0a1e8682"
I20260809 22:43:31.514984 23821 raft_consensus.cc:3037] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.515071 23666 raft_consensus.cc:3037] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.518271 23666 raft_consensus.cc:2454] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 1.
I20260809 22:43:31.518271 23821 raft_consensus.cc:2454] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 1.
I20260809 22:43:31.518667 23890 leader_election.cc:302] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:31.518967 24050 raft_consensus.cc:2781] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:31.519452 24050 raft_consensus.cc:686] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 LEADER]: Becoming Leader. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Running, Role: LEADER
I20260809 22:43:31.519939 24050 consensus_queue.cc:237] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.524276 23493 catalog_manager.cc:5495] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "69725a2e618f4dfe9c99c983eb1e1939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } health_report { overall_health: HEALTHY } } }
W20260809 22:43:31.533582 23870 tablet.cc:2369] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:43:31 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315409, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
W20260809 22:43:31.570657 24022 tablet.cc:2369] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:31.575181 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.561425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.21.27.190:36521)
0809 22:43:31.561603 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.561608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.562171 (+ 563us) server_negotiation.cc:407] Connection header received
0809 22:43:31.562380 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.562385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.562458 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.562573 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:31.563815 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.564537 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.565778 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.566198 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.566336 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.569698 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:31.569703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:31.569706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:31.569733 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:31.572499 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.573187 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.573191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.573200 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.573304 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:31.573652 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:31.573658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:31.573665 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:31.573940 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:31.574092 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.574567 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.574887 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":58}
I20260809 22:43:31.575181 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.561483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36521 (local address 127.0.0.1:58142)
0809 22:43:31.562062 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.562079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.562251 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.562595 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.562598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.563390 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:31.563675 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.563684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.564672 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.564678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.565631 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.565644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.566381 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.566387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.566553 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.567307 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:31.567325 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:31.569518 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:31.572632 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.572647 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.572654 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.573061 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.573406 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:31.573412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:31.573416 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:31.573551 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:31.574086 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:31.574094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:31.574259 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.574453 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.574759 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
I20260809 22:43:31.580070 23493 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58142:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:43:31.580997 23493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:31.588389 23644 tablet_service.cc:1467] Processing CreateTablet for tablet 62dcd7ca057d4ad9ad053bcec09c1c74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34421121ed324d59918eba422b4fd931]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:31.588815 23792 tablet_service.cc:1467] Processing CreateTablet for tablet 62dcd7ca057d4ad9ad053bcec09c1c74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34421121ed324d59918eba422b4fd931]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:31.589812 23792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62dcd7ca057d4ad9ad053bcec09c1c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.589978 23644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62dcd7ca057d4ad9ad053bcec09c1c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.597826 24042 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: Bootstrap starting.
I20260809 22:43:31.598187 24043 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: Bootstrap starting.
I20260809 22:43:31.599478 24042 tablet_bootstrap.cc:654] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.598932 23941 tablet_service.cc:1467] Processing CreateTablet for tablet 62dcd7ca057d4ad9ad053bcec09c1c74 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=34421121ed324d59918eba422b4fd931]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:43:31.599850 24043 tablet_bootstrap.cc:654] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.600024 23941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62dcd7ca057d4ad9ad053bcec09c1c74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.602229 24042 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: No bootstrap required, opened a new log
I20260809 22:43:31.602389 24042 ts_tablet_manager.cc:1397] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 22:43:31.602449 24043 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: No bootstrap required, opened a new log
I20260809 22:43:31.602617 24043 ts_tablet_manager.cc:1397] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 22:43:31.603235 24042 raft_consensus.cc:348] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.603472 24042 raft_consensus.cc:374] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.603571 24042 raft_consensus.cc:729] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3549fdf0dd9340ee948a67acd8bae650, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.603539 24043 raft_consensus.cc:348] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.603924 24043 raft_consensus.cc:374] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.603852 24042 consensus_queue.cc:260] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.604497 24042 ts_tablet_manager.cc:1428] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:43:31.604400 24049 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.604816 24049 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.604081 24043 raft_consensus.cc:729] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.605739 24043 consensus_queue.cc:260] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.606199 24048 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.606599 24048 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.607086 24043 ts_tablet_manager.cc:1428] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:43:31.607707 24046 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: Bootstrap starting.
I20260809 22:43:31.609086 24046 tablet_bootstrap.cc:654] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.611049 24046 tablet_bootstrap.cc:492] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: No bootstrap required, opened a new log
I20260809 22:43:31.611243 24046 ts_tablet_manager.cc:1397] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:43:31.611877 24046 raft_consensus.cc:348] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.612085 24046 raft_consensus.cc:374] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.612205 24046 raft_consensus.cc:729] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.612440 24046 consensus_queue.cc:260] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.612829 24050 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.613034 24046 ts_tablet_manager.cc:1428] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:43:31.613193 24050 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.615492 24048 raft_consensus.cc:482] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:31.615725 24048 raft_consensus.cc:504] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.617256 24048 leader_election.cc:288] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:31.626626 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.618119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:57437 (local address 127.21.27.130:38455)
0809 22:43:31.618302 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.618311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.618420 (+ 109us) server_negotiation.cc:407] Connection header received
0809 22:43:31.618656 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.618661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.618773 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.618859 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.619911 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.621006 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.623908 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.625159 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.625333 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.625408 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.626168 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.626326 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:43:31.626665 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.618074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:57329 (local address 127.21.27.131:36763)
0809 22:43:31.618280 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.618289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.618714 (+ 425us) server_negotiation.cc:407] Connection header received
0809 22:43:31.618894 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.618899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.618991 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.619092 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:31.619757 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.620871 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.623902 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.625157 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.625341 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.625408 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.626171 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.626326 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:43:31.627035 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.617633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:36763 (local address 127.21.27.129:57329)
0809 22:43:31.618614 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.618628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.618767 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.619107 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.619111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.619136 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.619600 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.619610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.620998 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.621002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.623742 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.623757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.625621 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.625626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.625874 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.626021 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.626120 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.626351 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":370,"thread_start_us":652,"threads_started":1}
I20260809 22:43:31.627506 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.617589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:38455 (local address 127.21.27.129:57437)
0809 22:43:31.618295 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.618316 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.618510 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.619043 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.619051 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.619079 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:31.619721 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.619737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.621136 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.621141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.623743 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.623757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.625602 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.625610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.625872 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.626020 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.626120 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.626351 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":243,"thread_start_us":390,"threads_started":1}
I20260809 22:43:31.627993 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62dcd7ca057d4ad9ad053bcec09c1c74" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939" is_pre_election: true
I20260809 22:43:31.628392 23962 raft_consensus.cc:2454] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:31.628979 23592 leader_election.cc:302] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:31.629261 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62dcd7ca057d4ad9ad053bcec09c1c74" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650" is_pre_election: true
I20260809 22:43:31.629439 24048 raft_consensus.cc:2781] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:31.629549 23821 raft_consensus.cc:2454] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:31.629590 24048 raft_consensus.cc:482] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:31.629690 24048 raft_consensus.cc:3037] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.632056 24048 raft_consensus.cc:504] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.632696 24048 leader_election.cc:288] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Requested vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:31.633157 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62dcd7ca057d4ad9ad053bcec09c1c74" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650"
I20260809 22:43:31.633378 23821 raft_consensus.cc:3037] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.633263 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62dcd7ca057d4ad9ad053bcec09c1c74" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939"
I20260809 22:43:31.633538 23962 raft_consensus.cc:3037] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:31.635821 23821 raft_consensus.cc:2454] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:31.636045 23962 raft_consensus.cc:2454] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:31.636263 23593 leader_election.cc:302] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3549fdf0dd9340ee948a67acd8bae650, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:31.636593 24048 raft_consensus.cc:2781] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:31.637058 24048 raft_consensus.cc:686] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 LEADER]: Becoming Leader. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Running, Role: LEADER
I20260809 22:43:31.637785 24048 consensus_queue.cc:237] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.641234 24065 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: TxnStatusTablet state changed. Reason: New leader ee37ae34072e4058b9e9958a0a1e8682. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.641741 24065 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:31.642526 24067 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:43:31.642840 24067 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:43:31.642907 23492 catalog_manager.cc:5495] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } health_report { overall_health: UNKNOWN } } }
I20260809 22:43:31.667389 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.661806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.21.27.129:45965)
0809 22:43:31.662021 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.662028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.662052 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:43:31.662162 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.662167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.662302 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.662483 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:31.662815 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.663672 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.665013 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.665438 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.665587 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.666101 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.666661 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.666818 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.666903 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.666987 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:43:31.667415 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.661531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45965 (local address 127.0.0.1:43980)
0809 22:43:31.661799 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.661820 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.661962 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.662493 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.662496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.662505 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:31.662666 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.662674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.663818 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.663826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.664865 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.664877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.665618 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.665624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.665802 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.665859 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.666775 (+ 916us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.666797 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.666883 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.667038 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:43:31.675693 23962 raft_consensus.cc:1261] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:31.676002 23821 raft_consensus.cc:1261] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:31.676839 24065 consensus_queue.cc:1035] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:31.677137 24048 consensus_queue.cc:1035] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:31.686142 24049 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: TxnStatusTablet state changed. Reason: New leader ee37ae34072e4058b9e9958a0a1e8682. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.686480 24049 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.691231 24065 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.691570 24065 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:31.700276 24049 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.700577 24049 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 3549fdf0dd9340ee948a67acd8bae650: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.704561 24050 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: TxnStatusTablet state changed. Reason: New leader ee37ae34072e4058b9e9958a0a1e8682. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.704928 24050 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.706485 24048 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.706792 24048 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P ee37ae34072e4058b9e9958a0a1e8682: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:43:31.710987 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.703573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.21.27.131:36763)
0809 22:43:31.703749 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.703756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.703997 (+ 241us) server_negotiation.cc:407] Connection header received
0809 22:43:31.704304 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.704310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.704412 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.704570 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:31.705327 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.706565 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.708551 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.709257 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.709390 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.709832 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.710333 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.710489 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.710578 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.710657 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:43:31.711989 24050 tablet_replica.cc:439] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } } }
I20260809 22:43:31.712276 24050 tablet_replica.cc:442] T 62dcd7ca057d4ad9ad053bcec09c1c74 P 69725a2e618f4dfe9c99c983eb1e1939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:43:31.729871 23981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.723775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45965 (local address 127.21.27.131:40711)
0809 22:43:31.723981 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.723998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.724094 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.724447 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.724449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.724458 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:31.724616 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.724623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.726034 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.726037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.727204 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.727216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.728311 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.728317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.728528 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.728608 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.729135 (+ 527us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.729149 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.729293 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.729476 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:43:31.729933 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.723894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:40711 (local address 127.21.27.129:45965)
0809 22:43:31.724067 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.724073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.724093 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:43:31.724246 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.724251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.724333 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.724432 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:31.724773 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.725917 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.727337 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.728068 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.728193 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.728840 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.728988 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.729079 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.729345 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.729457 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:43:31.743376 23821 raft_consensus.cc:1261] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Refusing update from remote peer 69725a2e618f4dfe9c99c983eb1e1939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:31.743678 23666 raft_consensus.cc:1261] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Refusing update from remote peer 69725a2e618f4dfe9c99c983eb1e1939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:31.744025 24052 consensus_queue.cc:1035] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:31.744421 24050 consensus_queue.cc:1035] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:31.783881 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.774743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:56715 (local address 127.21.27.131:36763)
0809 22:43:31.774928 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:31.774937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:31.776885 (+ 1948us) server_negotiation.cc:407] Connection header received
0809 22:43:31.777077 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:31.777085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:31.777169 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:31.777258 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:31.778078 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.779258 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.781631 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.782298 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.782398 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.783081 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.783241 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.783304 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:31.783527 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:43:31.783614 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 22:43:31.784842 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:31.774340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:36763 (local address 127.21.27.129:56715)
0809 22:43:31.776768 (+ 2428us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:31.776788 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:31.776935 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:31.777430 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:31.777435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:31.777447 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:31.777896 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.777907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.779464 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.779469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.781457 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:31.781473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:31.782609 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:31.782615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:31.782787 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:31.782859 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:31.783382 (+ 523us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:31.783392 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:43:31.784124 (+ 732us) client_negotiation.cc:240] Negotiation successful
0809 22:43:31.784285 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2173,"thread_start_us":2027,"threads_started":1}
W20260809 22:43:31.810040 23593 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.27.130:38455, user_credentials={real_user=slave}} blocked reactor thread for 44721.2us
I20260809 22:43:31.833331 23492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:31.833657 23492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:31.837803 23492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Sending DeleteTablet for 3 replicas of tablet 98727ebe4f9f4893b9e3e307e42a8670
I20260809 22:43:31.838937 23644 tablet_service.cc:1514] Processing DeleteTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43966
I20260809 22:43:31.838958 23792 tablet_service.cc:1514] Processing DeleteTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260809 22:43:31.840157 23941 tablet_service.cc:1514] Processing DeleteTablet for tablet 98727ebe4f9f4893b9e3e307e42a8670 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43914
I20260809 22:43:31.849582 23492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:31.851827 23492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:31.864284 23643 tablet_service.cc:1467] Processing CreateTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 (DEFAULT_TABLE table=test-table [id=fd3d04c08c1648849b30d9ada89743c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.864952 23643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f6db0618734aad8558bf4f0b3a2cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.867972 23940 tablet_service.cc:1467] Processing CreateTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 (DEFAULT_TABLE table=test-table [id=fd3d04c08c1648849b30d9ada89743c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.868510 23940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f6db0618734aad8558bf4f0b3a2cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.871712 23791 tablet_service.cc:1467] Processing CreateTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 (DEFAULT_TABLE table=test-table [id=fd3d04c08c1648849b30d9ada89743c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:31.872264 23791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f6db0618734aad8558bf4f0b3a2cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:31.874004 24043 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Bootstrap starting.
I20260809 22:43:31.875769 24043 tablet_bootstrap.cc:654] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.884895 24042 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Bootstrap starting.
I20260809 22:43:31.885126 24046 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Bootstrap starting.
I20260809 22:43:31.885856 24092 tablet_replica.cc:330] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: stopping tablet replica
I20260809 22:43:31.886482 24092 raft_consensus.cc:2229] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:31.886812 24042 tablet_bootstrap.cc:654] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.887382 24046 tablet_bootstrap.cc:654] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:31.887537 24092 raft_consensus.cc:2258] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:31.890693 24091 tablet_replica.cc:330] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: stopping tablet replica
I20260809 22:43:31.892037 24092 ts_tablet_manager.cc:1905] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:31.892409 24090 tablet_replica.cc:330] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: stopping tablet replica
I20260809 22:43:31.895977 24091 raft_consensus.cc:2229] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:31.896806 24090 raft_consensus.cc:2229] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:31.897416 24090 raft_consensus.cc:2258] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:31.897512 24046 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: No bootstrap required, opened a new log
I20260809 22:43:31.897697 24046 ts_tablet_manager.cc:1397] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.009s
I20260809 22:43:31.898066 24091 raft_consensus.cc:2258] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:31.898545 24046 raft_consensus.cc:348] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.898808 24046 raft_consensus.cc:374] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.898882 24046 raft_consensus.cc:729] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.899247 24043 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: No bootstrap required, opened a new log
I20260809 22:43:31.899159 24046 consensus_queue.cc:260] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.899411 24043 ts_tablet_manager.cc:1397] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.006s
I20260809 22:43:31.899865 24046 ts_tablet_manager.cc:1428] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:43:31.900452 24090 ts_tablet_manager.cc:1905] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:31.900979 24091 ts_tablet_manager.cc:1905] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:31.901026 24043 raft_consensus.cc:348] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.901314 24043 raft_consensus.cc:374] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.901412 24043 raft_consensus.cc:729] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.901681 24043 consensus_queue.cc:260] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.902390 24043 ts_tablet_manager.cc:1428] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:43:31.912593 24092 ts_tablet_manager.cc:1918] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:31.912734 24090 ts_tablet_manager.cc:1918] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:31.912961 24090 log.cc:1196] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/98727ebe4f9f4893b9e3e307e42a8670
I20260809 22:43:31.913550 24090 ts_tablet_manager.cc:1939] T 98727ebe4f9f4893b9e3e307e42a8670 P ee37ae34072e4058b9e9958a0a1e8682: Deleting consensus metadata
I20260809 22:43:31.915409 23477 catalog_manager.cc:4841] TS ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965): tablet 98727ebe4f9f4893b9e3e307e42a8670 (table test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]) successfully deleted
I20260809 22:43:31.916613 24042 tablet_bootstrap.cc:492] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: No bootstrap required, opened a new log
I20260809 22:43:31.916810 24042 ts_tablet_manager.cc:1397] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.010s
I20260809 22:43:31.917701 24042 raft_consensus.cc:348] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.917979 24042 raft_consensus.cc:374] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:31.918077 24042 raft_consensus.cc:729] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3549fdf0dd9340ee948a67acd8bae650, State: Initialized, Role: FOLLOWER
I20260809 22:43:31.918406 24042 consensus_queue.cc:260] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:31.919046 24042 ts_tablet_manager.cc:1428] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 22:43:31.919751 24092 log.cc:1196] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/98727ebe4f9f4893b9e3e307e42a8670
I20260809 22:43:31.920400 24092 ts_tablet_manager.cc:1939] T 98727ebe4f9f4893b9e3e307e42a8670 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting consensus metadata
I20260809 22:43:31.922348 23478 catalog_manager.cc:4841] TS 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763): tablet 98727ebe4f9f4893b9e3e307e42a8670 (table test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]) successfully deleted
I20260809 22:43:31.923213 24091 ts_tablet_manager.cc:1918] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:31.923388 24091 log.cc:1196] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/98727ebe4f9f4893b9e3e307e42a8670
I20260809 22:43:31.923921 24091 ts_tablet_manager.cc:1939] T 98727ebe4f9f4893b9e3e307e42a8670 P 3549fdf0dd9340ee948a67acd8bae650: Deleting consensus metadata
I20260809 22:43:31.925582 23479 catalog_manager.cc:4841] TS 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455): tablet 98727ebe4f9f4893b9e3e307e42a8670 (table test-table [id=409448a2560b4f7cbe0d8f7fa66bc475]) successfully deleted
I20260809 22:43:32.032546 24085 raft_consensus.cc:482] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:32.032919 24085 raft_consensus.cc:504] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:32.034554 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f6db0618734aad8558bf4f0b3a2cb3" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650" is_pre_election: true
I20260809 22:43:32.034649 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f6db0618734aad8558bf4f0b3a2cb3" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939" is_pre_election: true
I20260809 22:43:32.034839 23821 raft_consensus.cc:2454] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:32.034910 23962 raft_consensus.cc:2454] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:32.035416 23592 leader_election.cc:302] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:32.035758 24085 leader_election.cc:288] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:32.035789 24048 raft_consensus.cc:2781] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:32.036191 24048 raft_consensus.cc:482] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:32.036312 24048 raft_consensus.cc:3037] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:32.039621 24048 raft_consensus.cc:504] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:32.040788 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f6db0618734aad8558bf4f0b3a2cb3" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939"
I20260809 22:43:32.040854 23821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f6db0618734aad8558bf4f0b3a2cb3" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650"
I20260809 22:43:32.041005 23962 raft_consensus.cc:3037] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:32.041123 23821 raft_consensus.cc:3037] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:32.044497 23962 raft_consensus.cc:2454] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:32.044888 23821 raft_consensus.cc:2454] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:32.044978 23592 leader_election.cc:302] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:32.045300 24085 raft_consensus.cc:2781] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:32.045459 24085 raft_consensus.cc:686] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 LEADER]: Becoming Leader. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Running, Role: LEADER
I20260809 22:43:32.045787 24048 leader_election.cc:288] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Requested vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:32.045795 24085 consensus_queue.cc:237] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:32.049153 23492 catalog_manager.cc:5495] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } health_report { overall_health: UNKNOWN } } }
I20260809 22:43:32.098428 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:32.091759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.21.27.129:45965)
0809 22:43:32.091929 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:32.091935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:32.091955 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:43:32.092022 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:32.092027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:32.092112 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:32.092236 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:32.092821 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.094067 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.096130 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.096883 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.097017 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:32.097583 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:32.097762 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:32.097875 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:32.098011 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 22:43:32.098103 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:43:32.116164 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:32.109186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:42279 (local address 127.21.27.129:45965)
0809 22:43:32.109786 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:32.109792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:32.109809 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:43:32.109864 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:32.109868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:32.109950 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:32.110043 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:32.110516 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.111791 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.113874 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.114567 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.114767 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:32.115254 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:32.115416 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:32.115508 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:32.115707 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 22:43:32.115791 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":484}
I20260809 22:43:32.116171 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:32.109111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45965 (local address 127.21.27.129:42279)
0809 22:43:32.109407 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:32.109427 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:32.109555 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:32.110097 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:32.110103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:32.110117 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:43:32.110345 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:32.110364 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.111943 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.111949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:32.113708 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:32.113724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:32.114805 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:32.114812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:32.114995 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:32.115047 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:43:32.115538 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:43:32.115552 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 22:43:32.115669 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 22:43:32.115818 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 22:43:32.129061 23962 raft_consensus.cc:1261] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:32.130031 23821 raft_consensus.cc:1261] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:32.130720 24065 consensus_queue.cc:1035] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:32.131314 24048 consensus_queue.cc:1035] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:32.146965 24084 mvcc.cc:205] Tried to move back new op lower bound from 7316747928072998912 to 7316747927743315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316747928072998912}]
I20260809 22:43:32.159355 24072 mvcc.cc:205] Tried to move back new op lower bound from 7316747928072998912 to 7316747927743315968. Current Snapshot: MvccSnapshot[applied={T|T < 7316747928072998912}]
W20260809 22:43:32.178725 24084 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:43:33.797621 23492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:33.798005 23492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:33.802977 23492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Sending DeleteTablet for 3 replicas of tablet 75f6db0618734aad8558bf4f0b3a2cb3
I20260809 22:43:33.803848 23644 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:33 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43966
I20260809 22:43:33.804183 23792 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:33 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260809 22:43:33.804209 23940 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f6db0618734aad8558bf4f0b3a2cb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:33 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43914
I20260809 22:43:33.815429 24125 tablet_replica.cc:330] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: stopping tablet replica
I20260809 22:43:33.815968 24125 raft_consensus.cc:2229] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:33.816551 24125 raft_consensus.cc:2258] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:33.818085 24125 ts_tablet_manager.cc:1905] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:33.822467 24123 tablet_replica.cc:330] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: stopping tablet replica
I20260809 22:43:33.822839 24123 raft_consensus.cc:2229] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:33.823809 24123 raft_consensus.cc:2258] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:33.824784 24124 tablet_replica.cc:330] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: stopping tablet replica
I20260809 22:43:33.825048 24124 raft_consensus.cc:2229] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:33.825431 24123 ts_tablet_manager.cc:1905] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:33.825414 24124 raft_consensus.cc:2258] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:33.826781 24124 ts_tablet_manager.cc:1905] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:33.828352 24125 ts_tablet_manager.cc:1918] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:33.828613 24125 log.cc:1196] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/75f6db0618734aad8558bf4f0b3a2cb3
I20260809 22:43:33.829375 24125 ts_tablet_manager.cc:1939] T 75f6db0618734aad8558bf4f0b3a2cb3 P 3549fdf0dd9340ee948a67acd8bae650: Deleting consensus metadata
I20260809 22:43:33.830819 23479 catalog_manager.cc:4841] TS 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455): tablet 75f6db0618734aad8558bf4f0b3a2cb3 (table test-table [id=fd3d04c08c1648849b30d9ada89743c7]) successfully deleted
I20260809 22:43:33.833679 24124 ts_tablet_manager.cc:1918] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:33.833842 24124 log.cc:1196] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/75f6db0618734aad8558bf4f0b3a2cb3
I20260809 22:43:33.834316 24124 ts_tablet_manager.cc:1939] T 75f6db0618734aad8558bf4f0b3a2cb3 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting consensus metadata
I20260809 22:43:33.834578 24123 ts_tablet_manager.cc:1918] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:33.834792 24123 log.cc:1196] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/75f6db0618734aad8558bf4f0b3a2cb3
I20260809 22:43:33.835348 24123 ts_tablet_manager.cc:1939] T 75f6db0618734aad8558bf4f0b3a2cb3 P ee37ae34072e4058b9e9958a0a1e8682: Deleting consensus metadata
I20260809 22:43:33.835515 23478 catalog_manager.cc:4841] TS 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763): tablet 75f6db0618734aad8558bf4f0b3a2cb3 (table test-table [id=fd3d04c08c1648849b30d9ada89743c7]) successfully deleted
I20260809 22:43:33.836577 23477 catalog_manager.cc:4841] TS ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965): tablet 75f6db0618734aad8558bf4f0b3a2cb3 (table test-table [id=fd3d04c08c1648849b30d9ada89743c7]) successfully deleted
Aug 09 22:43:33 dist-test-slave-chsk krb5kdc[23440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:43:33 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:33.878316 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:33.861503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.21.27.190:36521)
0809 22:43:33.862226 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:33.862231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:33.862253 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:43:33.862436 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:33.862440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:33.862501 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:33.862594 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:33.864769 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:33.865528 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:33.867078 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:33.867541 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:33.867646 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:33.872266 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:33.872272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:33.872275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:33.872307 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:33.875807 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:33.876510 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:33.876514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:33.876516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:33.876698 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:33.877045 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:33.877049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:33.877051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:33.877340 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:33.877510 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:33.877799 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 22:43:33.877984 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":374,"threads_started":1}
I20260809 22:43:33.882352 23492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:33.882997 23492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:33.889912 23644 tablet_service.cc:1467] Processing CreateTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 (DEFAULT_TABLE table=test-table [id=7caf66fc159745c4beaa93c6200b4b8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:33.890141 23940 tablet_service.cc:1467] Processing CreateTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 (DEFAULT_TABLE table=test-table [id=7caf66fc159745c4beaa93c6200b4b8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:33.890431 23644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554cd13e9c9c4f5c9c53a35911e805d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:33.890380 23792 tablet_service.cc:1467] Processing CreateTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 (DEFAULT_TABLE table=test-table [id=7caf66fc159745c4beaa93c6200b4b8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:33.890650 23940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554cd13e9c9c4f5c9c53a35911e805d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:33.890915 23792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 554cd13e9c9c4f5c9c53a35911e805d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:33.897900 24134 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Bootstrap starting.
I20260809 22:43:33.901100 24134 tablet_bootstrap.cc:654] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:33.901177 24136 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Bootstrap starting.
I20260809 22:43:33.905372 24136 tablet_bootstrap.cc:654] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:33.905668 24135 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Bootstrap starting.
I20260809 22:43:33.908315 24134 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: No bootstrap required, opened a new log
I20260809 22:43:33.908493 24134 ts_tablet_manager.cc:1397] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260809 22:43:33.909034 24135 tablet_bootstrap.cc:654] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:33.909215 24134 raft_consensus.cc:348] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.909449 24134 raft_consensus.cc:374] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:33.909548 24134 raft_consensus.cc:729] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3549fdf0dd9340ee948a67acd8bae650, State: Initialized, Role: FOLLOWER
I20260809 22:43:33.909804 24134 consensus_queue.cc:260] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.911204 24134 ts_tablet_manager.cc:1428] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 22:43:33.914304 24135 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: No bootstrap required, opened a new log
I20260809 22:43:33.914481 24135 ts_tablet_manager.cc:1397] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 22:43:33.914604 24136 tablet_bootstrap.cc:492] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: No bootstrap required, opened a new log
I20260809 22:43:33.914765 24136 ts_tablet_manager.cc:1397] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260809 22:43:33.915338 24135 raft_consensus.cc:348] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.915560 24135 raft_consensus.cc:374] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:33.915685 24135 raft_consensus.cc:729] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Initialized, Role: FOLLOWER
I20260809 22:43:33.915632 24136 raft_consensus.cc:348] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.915895 24136 raft_consensus.cc:374] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:33.915998 24136 raft_consensus.cc:729] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Initialized, Role: FOLLOWER
I20260809 22:43:33.915953 24135 consensus_queue.cc:260] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.916285 24136 consensus_queue.cc:260] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:33.916577 24135 ts_tablet_manager.cc:1428] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:43:33.917591 24136 ts_tablet_manager.cc:1428] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:34.003904 24141 raft_consensus.cc:482] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:34.004317 24141 raft_consensus.cc:504] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.005127 24141 leader_election.cc:288] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965), 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455)
I20260809 22:43:34.005744 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554cd13e9c9c4f5c9c53a35911e805d6" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650" is_pre_election: true
I20260809 22:43:34.005813 23665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554cd13e9c9c4f5c9c53a35911e805d6" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee37ae34072e4058b9e9958a0a1e8682" is_pre_election: true
I20260809 22:43:34.006014 23820 raft_consensus.cc:2454] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 0.
I20260809 22:43:34.006114 23665 raft_consensus.cc:2454] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 0.
I20260809 22:43:34.006546 23892 leader_election.cc:302] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3549fdf0dd9340ee948a67acd8bae650, 69725a2e618f4dfe9c99c983eb1e1939; no voters:
I20260809 22:43:34.006891 24141 raft_consensus.cc:2781] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:34.007015 24141 raft_consensus.cc:482] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:34.007092 24141 raft_consensus.cc:3037] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.009253 24141 raft_consensus.cc:504] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.009933 24141 leader_election.cc:288] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 election: Requested vote from peers ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965), 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455)
I20260809 22:43:34.010437 23665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554cd13e9c9c4f5c9c53a35911e805d6" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee37ae34072e4058b9e9958a0a1e8682"
I20260809 22:43:34.010525 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "554cd13e9c9c4f5c9c53a35911e805d6" candidate_uuid: "69725a2e618f4dfe9c99c983eb1e1939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650"
I20260809 22:43:34.010684 23665 raft_consensus.cc:3037] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.010738 23820 raft_consensus.cc:3037] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.012872 23820 raft_consensus.cc:2454] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 1.
I20260809 22:43:34.012929 23665 raft_consensus.cc:2454] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69725a2e618f4dfe9c99c983eb1e1939 in term 1.
I20260809 22:43:34.013299 23892 leader_election.cc:302] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3549fdf0dd9340ee948a67acd8bae650, 69725a2e618f4dfe9c99c983eb1e1939; no voters:
I20260809 22:43:34.013615 24141 raft_consensus.cc:2781] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:34.014050 24141 raft_consensus.cc:686] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 LEADER]: Becoming Leader. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Running, Role: LEADER
I20260809 22:43:34.014501 24141 consensus_queue.cc:237] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.017241 23493 catalog_manager.cc:5495] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "69725a2e618f4dfe9c99c983eb1e1939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } health_report { overall_health: HEALTHY } } }
I20260809 22:43:34.038492 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:34.033500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.21.27.131:36763)
0809 22:43:34.034027 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:34.034038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:34.034057 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:43:34.034103 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:34.034107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:34.034161 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:34.034242 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:34.034583 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:34.035459 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:34.036678 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:34.037122 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:34.037211 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:34.037665 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:34.037772 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:34.037848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:34.038034 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 22:43:34.038093 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":310,"threads_started":1}
I20260809 22:43:34.047331 23820 raft_consensus.cc:1261] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Refusing update from remote peer 69725a2e618f4dfe9c99c983eb1e1939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:34.047521 23665 raft_consensus.cc:1261] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Refusing update from remote peer 69725a2e618f4dfe9c99c983eb1e1939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:34.047982 24142 consensus_queue.cc:1035] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:34.048364 24141 consensus_queue.cc:1035] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:34.053359 24106 mvcc.cc:205] Tried to move back new op lower bound from 7316747935932108800 to 7316747935805743104. Current Snapshot: MvccSnapshot[applied={T|T < 7316747935932108800}]
I20260809 22:43:34.091756 23493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:34.092021 23493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:34.094978 23493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Sending DeleteTablet for 3 replicas of tablet 554cd13e9c9c4f5c9c53a35911e805d6
I20260809 22:43:34.095981 23792 tablet_service.cc:1514] Processing DeleteTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:34 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260809 22:43:34.096037 23940 tablet_service.cc:1514] Processing DeleteTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:34 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43914
I20260809 22:43:34.096282 24125 tablet_replica.cc:330] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: stopping tablet replica
I20260809 22:43:34.096333 24124 tablet_replica.cc:330] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: stopping tablet replica
I20260809 22:43:34.096514 24124 raft_consensus.cc:2229] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:34.096573 24125 raft_consensus.cc:2229] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:34.096773 24125 raft_consensus.cc:2258] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:34.096859 24124 raft_consensus.cc:2258] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:34.096969 23493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 22:43:34.097260 23644 tablet_service.cc:1514] Processing DeleteTablet for tablet 554cd13e9c9c4f5c9c53a35911e805d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:34 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43966
I20260809 22:43:34.097546 24123 tablet_replica.cc:330] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: stopping tablet replica
I20260809 22:43:34.097745 24123 raft_consensus.cc:2229] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 22:43:34.097796 23493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:34.097925 24123 raft_consensus.cc:2258] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:34.097954 24124 ts_tablet_manager.cc:1905] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:34.097960 24125 ts_tablet_manager.cc:1905] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:34.099289 24123 ts_tablet_manager.cc:1905] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:34.104926 24124 ts_tablet_manager.cc:1918] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:34.105194 24124 log.cc:1196] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/554cd13e9c9c4f5c9c53a35911e805d6
I20260809 22:43:34.105731 24124 ts_tablet_manager.cc:1939] T 554cd13e9c9c4f5c9c53a35911e805d6 P 69725a2e618f4dfe9c99c983eb1e1939: Deleting consensus metadata
I20260809 22:43:34.105728 23940 tablet_service.cc:1467] Processing CreateTablet for tablet acece8c0b24c42ab9e5866d0bb64457b (DEFAULT_TABLE table=test-table [id=23d9a90bff674862a7790e028b2aa8a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:34.106067 23792 tablet_service.cc:1467] Processing CreateTablet for tablet acece8c0b24c42ab9e5866d0bb64457b (DEFAULT_TABLE table=test-table [id=23d9a90bff674862a7790e028b2aa8a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:34.106365 23940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acece8c0b24c42ab9e5866d0bb64457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:34.106571 24123 ts_tablet_manager.cc:1918] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:34.106735 24123 log.cc:1196] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/554cd13e9c9c4f5c9c53a35911e805d6
I20260809 22:43:34.106768 23644 tablet_service.cc:1467] Processing CreateTablet for tablet acece8c0b24c42ab9e5866d0bb64457b (DEFAULT_TABLE table=test-table [id=23d9a90bff674862a7790e028b2aa8a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:34.107218 24123 ts_tablet_manager.cc:1939] T 554cd13e9c9c4f5c9c53a35911e805d6 P ee37ae34072e4058b9e9958a0a1e8682: Deleting consensus metadata
I20260809 22:43:34.107213 23644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acece8c0b24c42ab9e5866d0bb64457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:34.108348 23477 catalog_manager.cc:4841] TS ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965): tablet 554cd13e9c9c4f5c9c53a35911e805d6 (table test-table [id=7caf66fc159745c4beaa93c6200b4b8e]) successfully deleted
I20260809 22:43:34.111212 24136 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Bootstrap starting.
I20260809 22:43:34.111562 23478 catalog_manager.cc:4841] TS 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763): tablet 554cd13e9c9c4f5c9c53a35911e805d6 (table test-table [id=7caf66fc159745c4beaa93c6200b4b8e]) successfully deleted
I20260809 22:43:34.111987 23792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acece8c0b24c42ab9e5866d0bb64457b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:34.113642 24136 tablet_bootstrap.cc:654] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:34.115253 24135 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Bootstrap starting.
I20260809 22:43:34.104768 24125 ts_tablet_manager.cc:1918] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:43:34.115509 24125 log.cc:1196] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/554cd13e9c9c4f5c9c53a35911e805d6
I20260809 22:43:34.116084 24125 ts_tablet_manager.cc:1939] T 554cd13e9c9c4f5c9c53a35911e805d6 P 3549fdf0dd9340ee948a67acd8bae650: Deleting consensus metadata
I20260809 22:43:34.116979 24136 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: No bootstrap required, opened a new log
I20260809 22:43:34.117208 24136 ts_tablet_manager.cc:1397] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 22:43:34.117390 24135 tablet_bootstrap.cc:654] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:34.117341 23479 catalog_manager.cc:4841] TS 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455): tablet 554cd13e9c9c4f5c9c53a35911e805d6 (table test-table [id=7caf66fc159745c4beaa93c6200b4b8e]) successfully deleted
I20260809 22:43:34.118067 24136 raft_consensus.cc:348] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.118422 24136 raft_consensus.cc:374] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:34.118506 24136 raft_consensus.cc:729] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69725a2e618f4dfe9c99c983eb1e1939, State: Initialized, Role: FOLLOWER
I20260809 22:43:34.119498 24136 consensus_queue.cc:260] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.120321 24136 ts_tablet_manager.cc:1428] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:34.120563 24135 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: No bootstrap required, opened a new log
I20260809 22:43:34.120820 24135 ts_tablet_manager.cc:1397] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 22:43:34.122208 24135 raft_consensus.cc:348] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.122666 24135 raft_consensus.cc:374] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:34.122891 24135 raft_consensus.cc:729] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Initialized, Role: FOLLOWER
I20260809 22:43:34.123329 24135 consensus_queue.cc:260] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.124418 24134 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Bootstrap starting.
I20260809 22:43:34.124917 24135 ts_tablet_manager.cc:1428] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:43:34.126864 24134 tablet_bootstrap.cc:654] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:34.129752 24134 tablet_bootstrap.cc:492] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: No bootstrap required, opened a new log
I20260809 22:43:34.130064 24134 ts_tablet_manager.cc:1397] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 22:43:34.131021 24134 raft_consensus.cc:348] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.131457 24134 raft_consensus.cc:374] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:34.131618 24134 raft_consensus.cc:729] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3549fdf0dd9340ee948a67acd8bae650, State: Initialized, Role: FOLLOWER
I20260809 22:43:34.131945 24134 consensus_queue.cc:260] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.132658 24134 ts_tablet_manager.cc:1428] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:43:34.226859 24105 raft_consensus.cc:482] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:34.227202 24105 raft_consensus.cc:504] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.228021 24105 leader_election.cc:288] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:34.228669 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acece8c0b24c42ab9e5866d0bb64457b" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650" is_pre_election: true
I20260809 22:43:34.228736 23961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acece8c0b24c42ab9e5866d0bb64457b" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939" is_pre_election: true
I20260809 22:43:34.228951 23820 raft_consensus.cc:2454] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:34.229151 23961 raft_consensus.cc:2454] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 0.
I20260809 22:43:34.229527 23593 leader_election.cc:302] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3549fdf0dd9340ee948a67acd8bae650, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:34.229879 24105 raft_consensus.cc:2781] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:34.230036 24105 raft_consensus.cc:482] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:34.230111 24105 raft_consensus.cc:3037] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.232852 24105 raft_consensus.cc:504] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.233485 24105 leader_election.cc:288] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Requested vote from peers 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455), 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763)
I20260809 22:43:34.233996 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acece8c0b24c42ab9e5866d0bb64457b" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3549fdf0dd9340ee948a67acd8bae650"
I20260809 22:43:34.234094 23961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acece8c0b24c42ab9e5866d0bb64457b" candidate_uuid: "ee37ae34072e4058b9e9958a0a1e8682" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69725a2e618f4dfe9c99c983eb1e1939"
I20260809 22:43:34.234230 23820 raft_consensus.cc:3037] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.234426 23961 raft_consensus.cc:3037] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:34.237196 23961 raft_consensus.cc:2454] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:34.237294 23820 raft_consensus.cc:2454] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee37ae34072e4058b9e9958a0a1e8682 in term 1.
I20260809 22:43:34.237722 23592 leader_election.cc:302] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69725a2e618f4dfe9c99c983eb1e1939, ee37ae34072e4058b9e9958a0a1e8682; no voters:
I20260809 22:43:34.238068 24105 raft_consensus.cc:2781] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:34.238297 24105 raft_consensus.cc:686] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 1 LEADER]: Becoming Leader. State: Replica: ee37ae34072e4058b9e9958a0a1e8682, State: Running, Role: LEADER
I20260809 22:43:34.238703 24105 consensus_queue.cc:237] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } }
I20260809 22:43:34.242136 23492 catalog_manager.cc:5495] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "ee37ae34072e4058b9e9958a0a1e8682" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee37ae34072e4058b9e9958a0a1e8682" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 } health_report { overall_health: UNKNOWN } } }
I20260809 22:43:34.295929 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:34.288877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.21.27.129:45965)
0809 22:43:34.289958 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:34.289964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:34.290006 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:43:34.290072 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:34.290077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:34.290184 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:34.290268 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:34.290739 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:34.292049 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:34.293741 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:34.294263 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:34.294407 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:34.294950 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:34.295113 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:34.295276 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:34.295449 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 22:43:34.295512 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":705,"threads_started":1}
I20260809 22:43:34.305709 23961 raft_consensus.cc:1261] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:34.305820 23820 raft_consensus.cc:1261] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Refusing update from remote peer ee37ae34072e4058b9e9958a0a1e8682: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:34.306424 24101 consensus_queue.cc:1035] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69725a2e618f4dfe9c99c983eb1e1939" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 36763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:34.306818 24110 consensus_queue.cc:1035] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3549fdf0dd9340ee948a67acd8bae650" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:43:34.329165 24103 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:43:35.914295 23492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:35.914651 23492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:35.919610 23492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9ec85978630467f8727e5201d63d9ba: Sending DeleteTablet for 3 replicas of tablet acece8c0b24c42ab9e5866d0bb64457b
I20260809 22:43:35.920666 23644 tablet_service.cc:1514] Processing DeleteTablet for tablet acece8c0b24c42ab9e5866d0bb64457b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:35 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43966
I20260809 22:43:35.920728 23792 tablet_service.cc:1514] Processing DeleteTablet for tablet acece8c0b24c42ab9e5866d0bb64457b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:35 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36850
I20260809 22:43:35.920907 23940 tablet_service.cc:1514] Processing DeleteTablet for tablet acece8c0b24c42ab9e5866d0bb64457b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:35 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43914
I20260809 22:43:35.921314 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23571
I20260809 22:43:35.936444 24180 tablet_replica.cc:330] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: stopping tablet replica
I20260809 22:43:35.937496 24180 raft_consensus.cc:2229] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:35.938306 24180 raft_consensus.cc:2258] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:35.938372 24182 tablet_replica.cc:330] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: stopping tablet replica
I20260809 22:43:35.938591 24182 raft_consensus.cc:2229] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:35.939422 24181 tablet_replica.cc:330] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: stopping tablet replica
I20260809 22:43:35.939591 24182 raft_consensus.cc:2258] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:35.939765 24181 raft_consensus.cc:2229] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:35.940233 24181 raft_consensus.cc:2258] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:35.940683 24180 ts_tablet_manager.cc:1905] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:35.940855 24182 ts_tablet_manager.cc:1905] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:35.941360 24181 ts_tablet_manager.cc:1905] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:35.949476 24182 ts_tablet_manager.cc:1918] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:35.949566 24181 ts_tablet_manager.cc:1918] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:35.949776 24181 log.cc:1196] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/acece8c0b24c42ab9e5866d0bb64457b
I20260809 22:43:35.949795 24182 log.cc:1196] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/acece8c0b24c42ab9e5866d0bb64457b
I20260809 22:43:35.950409 24181 ts_tablet_manager.cc:1939] T acece8c0b24c42ab9e5866d0bb64457b P ee37ae34072e4058b9e9958a0a1e8682: Deleting consensus metadata
I20260809 22:43:35.950374 24180 ts_tablet_manager.cc:1918] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:43:35.950562 24180 log.cc:1196] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/acece8c0b24c42ab9e5866d0bb64457b
I20260809 22:43:35.950646 24182 ts_tablet_manager.cc:1939] T acece8c0b24c42ab9e5866d0bb64457b P 69725a2e618f4dfe9c99c983eb1e1939: Deleting consensus metadata
I20260809 22:43:35.952138 23478 catalog_manager.cc:4841] TS 69725a2e618f4dfe9c99c983eb1e1939 (127.21.27.131:36763): tablet acece8c0b24c42ab9e5866d0bb64457b (table test-table [id=23d9a90bff674862a7790e028b2aa8a4]) successfully deleted
I20260809 22:43:35.952343 24180 ts_tablet_manager.cc:1939] T acece8c0b24c42ab9e5866d0bb64457b P 3549fdf0dd9340ee948a67acd8bae650: Deleting consensus metadata
I20260809 22:43:35.952579 23477 catalog_manager.cc:4841] TS ee37ae34072e4058b9e9958a0a1e8682 (127.21.27.129:45965): tablet acece8c0b24c42ab9e5866d0bb64457b (table test-table [id=23d9a90bff674862a7790e028b2aa8a4]) successfully deleted
I20260809 22:43:35.953467 23479 catalog_manager.cc:4841] TS 3549fdf0dd9340ee948a67acd8bae650 (127.21.27.130:38455): tablet acece8c0b24c42ab9e5866d0bb64457b (table test-table [id=23d9a90bff674862a7790e028b2aa8a4]) successfully deleted
Aug 09 22:43:35 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:35.967330 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:35.924842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.21.27.129:45965)
0809 22:43:35.938168 (+ 13326us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:35.938178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:35.938227 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:43:35.938307 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:35.938315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:35.938459 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:35.938627 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:35.941461 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:35.943113 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:35.945459 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:35.946516 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:35.946673 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:35.958654 (+ 11981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:35.958679 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:35.958696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:35.958730 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:35.962920 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:35.963761 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:35.963770 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:35.963774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:35.964014 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:35.964593 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:35.964600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:35.964617 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:35.965513 (+ 896us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:35.965949 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:35.966207 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 22:43:35.966733 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13079,"thread_start_us":12781,"threads_started":1}
W20260809 22:43:35.968087 23713 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44012
W20260809 22:43:35.968933 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23571: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:35.969228 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23571
I20260809 22:43:35.998989 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23722
Aug 09 22:43:36 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:36.016294 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:35.999868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.21.27.130:38455)
0809 22:43:36.000710 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:36.000715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:36.000733 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:43:36.000786 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:36.000790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:36.000849 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:36.000966 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:36.002950 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:36.004095 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:36.005641 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:36.006079 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:36.006189 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:36.010448 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:36.010464 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:36.010481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:36.010518 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:36.013714 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:36.014417 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:36.014426 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:36.014428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:36.014562 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:36.014885 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:36.014888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:36.014890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:36.015346 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:36.015610 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:36.015843 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 22:43:36.016017 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":478,"threads_started":1}
W20260809 22:43:36.016973 23863 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36856
W20260809 22:43:36.017616 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:36.017758 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23722
I20260809 22:43:36.038298 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23873
Aug 09 22:43:36 dist-test-slave-chsk krb5kdc[23440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:36.054379 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:36.039077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.21.27.131:36763)
0809 22:43:36.039682 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:36.039688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:36.039704 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:43:36.039761 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:36.039764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:36.039835 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:36.039929 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:36.041291 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:36.042086 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:36.043468 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:36.043874 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:36.043977 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:36.048462 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:36.048478 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:36.048494 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:36.048531 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:36.051679 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:36.052381 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:36.052387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:36.052390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:36.052559 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:36.052912 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:36.052915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:36.052917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:36.053397 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:36.053680 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:36.053932 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 22:43:36.054108 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":365,"threads_started":1}
W20260809 22:43:36.055176 24016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43938
W20260809 22:43:36.055857 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23873: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:36.056053 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23873
I20260809 22:43:36.081467 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 23454
I20260809 22:43:36.082733 23538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:36.200141 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 23454
2026-08-09T22:43:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8325 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:36 dist-test-slave-chsk krb5kdc[24190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:36 dist-test-slave-chsk krb5kdc[24190](info): set up 2 sockets
Aug 09 22:43:36 dist-test-slave-chsk krb5kdc[24190](info): commencing operation
krb5kdc: starting...
W20260809 22:43:37.374675 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:37 dist-test-slave-chsk krb5kdc[24190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:37.586385 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38843
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:38843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:37.909140 24204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:37.909436 24204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:37.909607 24204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:37.916952 24204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:37.917061 24204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:37.917168 24204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:37.917248 24204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:37.917309 24204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:37.927248 24204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:38843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:37.928543 24204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:37.929857 24204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:37.941752 24210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:37.941704 24209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:37.941754 24212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:37.942363 24204 server_base.cc:1034] running on GCE node
I20260809 22:43:37.943128 24204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:37.944458 24204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:37.945693 24204 hybrid_clock.cc:648] HybridClock initialized: now 1786315417945637 us; error 34 us; skew 500 ppm
I20260809 22:43:37.946002 24204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:37 dist-test-slave-chsk krb5kdc[24190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315417, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:37.949069 24204 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:37.950397 24204 webserver.cc:460] Webserver started at http://127.21.27.190:42191/ using document root <none> and password file <none>
I20260809 22:43:37.950989 24204 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:37.951083 24204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:37.951398 24204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:37.954226 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "ec66580b3f0c4b888d82357bc8ec525f"
format_stamp: "Formatted at 2026-08-09 22:43:37 on dist-test-slave-chsk"
server_key: "f11959a19dc5e1db7089b5dd2c72d924"
server_key_iv: "2549dda3804a4edd95acdaccda0ef87f"
server_key_version: "encryptionkey@0"
I20260809 22:43:37.954972 24204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "ec66580b3f0c4b888d82357bc8ec525f"
format_stamp: "Formatted at 2026-08-09 22:43:37 on dist-test-slave-chsk"
server_key: "f11959a19dc5e1db7089b5dd2c72d924"
server_key_iv: "2549dda3804a4edd95acdaccda0ef87f"
server_key_version: "encryptionkey@0"
I20260809 22:43:37.959960 24204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 22:43:37.963393 24221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:37.964561 24204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:43:37.964740 24204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "ec66580b3f0c4b888d82357bc8ec525f"
format_stamp: "Formatted at 2026-08-09 22:43:37 on dist-test-slave-chsk"
server_key: "f11959a19dc5e1db7089b5dd2c72d924"
server_key_iv: "2549dda3804a4edd95acdaccda0ef87f"
server_key_version: "encryptionkey@0"
I20260809 22:43:37.964934 24204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:37.978693 24204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:37.982558 24204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:37.982839 24204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:38.001451 24204 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:38843
I20260809 22:43:38.001477 24276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:38843 every 8 connection(s)
I20260809 22:43:38.002998 24204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:38.006675 24277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:38.007210 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24204
I20260809 22:43:38.007462 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:38.008158 21614 external_mini_cluster.cc:1428] Setting key db33738bb7efcbf15aa39ff70658f30e
I20260809 22:43:38.020713 24277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f: Bootstrap starting.
Aug 09 22:43:38 dist-test-slave-chsk krb5kdc[24190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:38.024395 24277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:38.025604 24277 log.cc:824] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:38.028510 24277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f: No bootstrap required, opened a new log
I20260809 22:43:38.031397 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:38.010166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.21.27.190:38843)
0809 22:43:38.011047 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:38.011058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:38.011095 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:43:38.012777 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:38.012801 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:38.013152 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:38.013481 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:38.015344 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.016662 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.018502 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.019238 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.019389 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:38.024093 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:38.024102 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:38.024115 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:38.024156 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:38.027403 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:38.028080 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:38.028088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:38.028093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:38.028257 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:38.028644 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:38.028651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:38.028656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:38.029159 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:38.029489 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:38.029908 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 22:43:38.030345 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":376,"threads_started":1}
I20260809 22:43:38.036597 24277 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } }
I20260809 22:43:38.037066 24277 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:38.037199 24277 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec66580b3f0c4b888d82357bc8ec525f, State: Initialized, Role: FOLLOWER
I20260809 22:43:38.037796 24277 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } }
I20260809 22:43:38.038046 24277 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:38.038208 24277 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:38.038379 24277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:38.040630 24277 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } }
I20260809 22:43:38.041118 24277 leader_election.cc:302] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec66580b3f0c4b888d82357bc8ec525f; no voters:
I20260809 22:43:38.041733 24277 leader_election.cc:288] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:38.041898 24282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:38.043033 24282 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [term 1 LEADER]: Becoming Leader. State: Replica: ec66580b3f0c4b888d82357bc8ec525f, State: Running, Role: LEADER
I20260809 22:43:38.043583 24282 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } }
I20260809 22:43:38.043917 24277 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:38.046587 24285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec66580b3f0c4b888d82357bc8ec525f. Latest consensus state: current_term: 1 leader_uuid: "ec66580b3f0c4b888d82357bc8ec525f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } } }
I20260809 22:43:38.047016 24285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:38.046607 24284 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec66580b3f0c4b888d82357bc8ec525f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec66580b3f0c4b888d82357bc8ec525f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38843 } } }
I20260809 22:43:38.047982 24284 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:38.048844 24291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:38.054056 24291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:38.062958 24291 catalog_manager.cc:1347] Generated new cluster ID: d351ca3e595f4230acf2a143c0d736ee
I20260809 22:43:38.063081 24291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:38.101940 24291 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:38.103696 24291 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:38.109802 24291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec66580b3f0c4b888d82357bc8ec525f: Generated new TSK 0
I20260809 22:43:38.110975 24291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:38.166863 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:38.508075 24306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:38.508360 24306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:38.508530 24306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:38.515900 24306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:38.516039 24306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:38.516409 24306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:38.527642 24306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:38.528963 24306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:38.530242 24306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:38.543946 24311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:38.543825 24314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:38.543876 24312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:38.544735 24306 server_base.cc:1034] running on GCE node
I20260809 22:43:38.545454 24306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:38.546365 24306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:38.547556 24306 hybrid_clock.cc:648] HybridClock initialized: now 1786315418547499 us; error 52 us; skew 500 ppm
I20260809 22:43:38.547920 24306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:38 dist-test-slave-chsk krb5kdc[24190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315418, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:38.551326 24306 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:38.552770 24306 webserver.cc:460] Webserver started at http://127.21.27.129:32941/ using document root <none> and password file <none>
I20260809 22:43:38.553447 24306 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:38.553586 24306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:38.553929 24306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:38.556906 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "18a70f0a2ed74361a264df3481553334"
format_stamp: "Formatted at 2026-08-09 22:43:38 on dist-test-slave-chsk"
server_key: "5dbedcef799e1b0ebdfe2b6115cb1b6c"
server_key_iv: "d49263782672f12519581643b080c15c"
server_key_version: "encryptionkey@0"
I20260809 22:43:38.557719 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "18a70f0a2ed74361a264df3481553334"
format_stamp: "Formatted at 2026-08-09 22:43:38 on dist-test-slave-chsk"
server_key: "5dbedcef799e1b0ebdfe2b6115cb1b6c"
server_key_iv: "d49263782672f12519581643b080c15c"
server_key_version: "encryptionkey@0"
I20260809 22:43:38.562935 24306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:43:38.566380 24322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:38.567773 24306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:38.567942 24306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "18a70f0a2ed74361a264df3481553334"
format_stamp: "Formatted at 2026-08-09 22:43:38 on dist-test-slave-chsk"
server_key: "5dbedcef799e1b0ebdfe2b6115cb1b6c"
server_key_iv: "d49263782672f12519581643b080c15c"
server_key_version: "encryptionkey@0"
I20260809 22:43:38.568143 24306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:38.590448 24306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:38.594553 24306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:38.594867 24306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:38.595920 24306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:38.597282 24306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:38.597393 24306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:38.597539 24306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:38.597615 24306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:38.632473 24306 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:35787
I20260809 22:43:38.632568 24443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:35787 every 8 connection(s)
I20260809 22:43:38.633849 24306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:38.640337 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24306
I20260809 22:43:38.640581 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:38.641479 21614 external_mini_cluster.cc:1428] Setting key 7794f6c553b4312497d4014b3fe13146
Aug 09 22:43:38 dist-test-slave-chsk krb5kdc[24190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315418, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:38.655555 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:38.636486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:52089 (local address 127.21.27.190:38843)
0809 22:43:38.636915 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:38.636920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:38.637975 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 22:43:38.640011 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:38.640016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:38.640089 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:38.640203 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:38.642482 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.643329 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.645063 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.645532 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.645666 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:38.649430 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:38.649436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:38.649440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:38.649475 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:38.652389 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:38.653141 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:38.653148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:38.653151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:38.653281 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:38.653727 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:38.653732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:38.653736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:38.654020 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:38.654182 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:38.654900 (+ 718us) server_negotiation.cc:299] Negotiation successful
0809 22:43:38.655174 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":205,"threads_started":1}
I20260809 22:43:38.656582 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:38.637061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:38843 (local address 127.21.27.129:52089)
0809 22:43:38.637802 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:38.637849 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:38.639663 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:38.640516 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:38.640531 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:38.641429 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:38.642280 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:38.642307 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.643500 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.643508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:38.644904 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:38.644917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:38.645784 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:38.645792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:38.646018 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:38.646724 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:38.646749 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:38.649162 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:38.652537 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:38.652546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:38.652564 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:38.653017 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:38.653396 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:38.653401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:38.653405 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:38.653607 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:38.654155 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:38.654162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:38.654520 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 22:43:38.654801 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 22:43:38.655387 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":287,"threads_started":1}
I20260809 22:43:38.658355 24444 heartbeater.cc:347] Connected to a master server at 127.21.27.190:38843
I20260809 22:43:38.658672 24444 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:38.659444 24444 heartbeater.cc:511] Master 127.21.27.190:38843 requested a full tablet report, sending...
I20260809 22:43:38.662019 24239 ts_manager.cc:194] Registered new tserver with Master: 18a70f0a2ed74361a264df3481553334 (127.21.27.129:35787)
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:38.663862 24239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:52089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:38.690287 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:39.020709 24452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:39.020982 24452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:39.021075 24452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:39.028404 24452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:39.028523 24452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:39.028640 24452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:39.038443 24452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:39.039855 24452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:39.041157 24452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:39.054339 24458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:39.054409 24457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:39.054332 24460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:39.055346 24452 server_base.cc:1034] running on GCE node
I20260809 22:43:39.056094 24452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:39.057212 24452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:39.058440 24452 hybrid_clock.cc:648] HybridClock initialized: now 1786315419058381 us; error 55 us; skew 500 ppm
I20260809 22:43:39.058761 24452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:39 dist-test-slave-chsk krb5kdc[24190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:39.062062 24452 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:39.063690 24452 webserver.cc:460] Webserver started at http://127.21.27.130:35953/ using document root <none> and password file <none>
I20260809 22:43:39.064388 24452 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:39.064535 24452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:39.064852 24452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:39.067938 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "3c72d784c01a47ad91a5934087e0e5c7"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "3c287892881ad3f7cf7473f417c33fd0"
server_key_iv: "7156560ef6298b8061f23f64638ce1bc"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.068745 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "3c72d784c01a47ad91a5934087e0e5c7"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "3c287892881ad3f7cf7473f417c33fd0"
server_key_iv: "7156560ef6298b8061f23f64638ce1bc"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.074326 24452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:39.077873 24468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.079427 24452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:43:39.079609 24452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "3c72d784c01a47ad91a5934087e0e5c7"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "3c287892881ad3f7cf7473f417c33fd0"
server_key_iv: "7156560ef6298b8061f23f64638ce1bc"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.079818 24452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:39.125075 24452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:39.129567 24452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:39.129926 24452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:39.131049 24452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:39.132432 24452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:39.132506 24452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.132632 24452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:39.132695 24452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.166555 24452 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:45135
I20260809 22:43:39.166606 24587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:45135 every 8 connection(s)
I20260809 22:43:39.167917 24452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:39.172359 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24452
I20260809 22:43:39.172585 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:39.173087 21614 external_mini_cluster.cc:1428] Setting key 160252b8a230f9dde55e59de3de915fa
Aug 09 22:43:39 dist-test-slave-chsk krb5kdc[24190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:39.193475 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:39.170421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:36855 (local address 127.21.27.190:38843)
0809 22:43:39.170880 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:39.170884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:39.171721 (+ 837us) server_negotiation.cc:407] Connection header received
0809 22:43:39.173765 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:39.173775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:39.173873 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:39.174013 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:39.175753 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.176443 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.178957 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.179478 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.179579 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:39.184491 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:39.184496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:39.184500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:39.184535 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:39.188187 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:39.189347 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:39.189356 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:39.189362 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:39.189546 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:39.190374 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:39.190382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:39.190389 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:39.190812 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:39.190970 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:39.192213 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0809 22:43:39.192720 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":257,"threads_started":1}
I20260809 22:43:39.194216 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:39.170805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:38843 (local address 127.21.27.130:36855)
0809 22:43:39.171533 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:39.171575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:39.173487 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:39.174266 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:39.174274 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:39.174786 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:39.175568 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:39.175594 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.176603 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.176612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:39.178796 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:39.178814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.179708 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.179718 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:39.180062 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:39.180815 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:39.180848 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:39.184236 (+ 3388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:39.188407 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:39.188418 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:39.188438 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:39.189165 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:39.189734 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:39.189769 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:39.189776 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:39.190202 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:39.191052 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:39.191064 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:39.191607 (+ 543us) client_negotiation.cc:769] Sending connection context
0809 22:43:39.191954 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 22:43:39.192577 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":352,"threads_started":1}
I20260809 22:43:39.196105 24588 heartbeater.cc:347] Connected to a master server at 127.21.27.190:38843
I20260809 22:43:39.196477 24588 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
I20260809 22:43:39.197432 24588 heartbeater.cc:511] Master 127.21.27.190:38843 requested a full tablet report, sending...
I20260809 22:43:39.199277 24239 ts_manager.cc:194] Registered new tserver with Master: 3c72d784c01a47ad91a5934087e0e5c7 (127.21.27.130:45135)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
I20260809 22:43:39.200394 24239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:36855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:39.235409 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:39.557467 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:39.557725 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:39.557817 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:39.564898 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:39.564991 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:39.565106 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:39.574904 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:38843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:39.576100 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:39.577450 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:39.590286 24601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:39.590412 24604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:39.590435 24602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:39.591570 24595 server_base.cc:1034] running on GCE node
I20260809 22:43:39.592283 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:39.593199 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:39.594404 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786315419594365 us; error 31 us; skew 500 ppm
I20260809 22:43:39.594713 24595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:39 dist-test-slave-chsk krb5kdc[24190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:39.597818 24595 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:39.599274 24595 webserver.cc:460] Webserver started at http://127.21.27.131:37069/ using document root <none> and password file <none>
I20260809 22:43:39.599898 24595 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:39.600006 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:39.600263 24595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:39.603017 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "94a6ef9b771340e99559cedd36e8bc48"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "e1349c9c060f8a282c5cc0b8edf07acf"
server_key_iv: "f9e7ab6498300d96d2eba15eafa775d1"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.603820 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "94a6ef9b771340e99559cedd36e8bc48"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "e1349c9c060f8a282c5cc0b8edf07acf"
server_key_iv: "f9e7ab6498300d96d2eba15eafa775d1"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.608845 24595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:39.612138 24613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.613617 24595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:39.613751 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "94a6ef9b771340e99559cedd36e8bc48"
format_stamp: "Formatted at 2026-08-09 22:43:39 on dist-test-slave-chsk"
server_key: "e1349c9c060f8a282c5cc0b8edf07acf"
server_key_iv: "f9e7ab6498300d96d2eba15eafa775d1"
server_key_version: "encryptionkey@0"
I20260809 22:43:39.613915 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:39.631552 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:39.635911 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:39.636220 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:39.637302 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:39.638693 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:39.638767 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.638947 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:39.639000 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:39.667260 24444 heartbeater.cc:503] Master 127.21.27.190:38843 was elected leader, sending a full tablet report...
I20260809 22:43:39.674185 24595 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:34153
I20260809 22:43:39.674265 24730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:34153 every 8 connection(s)
I20260809 22:43:39.675477 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:39.676713 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24595
I20260809 22:43:39.676918 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:39.677486 21614 external_mini_cluster.cc:1428] Setting key cb1eb6b62c25a0020676ea92c7da50e5
Aug 09 22:43:39 dist-test-slave-chsk krb5kdc[24190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:39.694896 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:39.678137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:52797 (local address 127.21.27.190:38843)
0809 22:43:39.678339 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:39.678347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:39.679878 (+ 1531us) server_negotiation.cc:407] Connection header received
0809 22:43:39.681649 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:39.681653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:39.681717 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:39.681797 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:39.683230 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.684087 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.685542 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.685913 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.686005 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:39.689206 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:39.689211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:39.689215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:39.689246 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:39.691951 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:39.692683 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:39.692689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:39.692699 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:39.692813 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:39.693239 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:39.693244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:39.693249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:39.693512 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:39.693692 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:39.694322 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 22:43:39.694544 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:43:39.695818 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:39.678751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:38843 (local address 127.21.27.131:52797)
0809 22:43:39.679714 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:39.679766 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:39.681418 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:39.681992 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:39.682012 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:39.682520 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:39.683035 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:39.683051 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.684216 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.684220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:39.685420 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:39.685431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:39.686092 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:39.686096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:39.686291 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:39.686873 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:39.686898 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:39.689013 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:39.692101 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:39.692109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:39.692131 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:39.692510 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:39.692929 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:39.692935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:39.692938 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:39.693118 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:39.693659 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:39.693667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:39.693961 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 22:43:39.694209 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 22:43:39.694685 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":466,"threads_started":1}
I20260809 22:43:39.697103 24731 heartbeater.cc:347] Connected to a master server at 127.21.27.190:38843
I20260809 22:43:39.697394 24731 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:39.698055 24731 heartbeater.cc:511] Master 127.21.27.190:38843 requested a full tablet report, sending...
I20260809 22:43:39.699436 24239 ts_manager.cc:194] Registered new tserver with Master: 94a6ef9b771340e99559cedd36e8bc48 (127.21.27.131:34153)
I20260809 22:43:39.700213 24239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:52797
I20260809 22:43:39.704857 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:43:39.740223 24590 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:43:39.737762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.21.27.190:38843)
0809 22:43:39.737934 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:39.737939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:39.738369 (+ 430us) server_negotiation.cc:407] Connection header received
0809 22:43:39.738487 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:39.738492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:39.738564 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:39.738663 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:39.739981 (+ 1318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35532: BlockingRecv error: recv got EOF from 127.0.0.1:35532 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:43:39.743680 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24306
W20260809 22:43:39.746456 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24306: Not authorized: Client connection negotiation failed: client connection to 127.21.27.129:35787: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:43:39.746637 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24306
I20260809 22:43:39.764207 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24452
W20260809 22:43:39.766760 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24452: Not authorized: Client connection negotiation failed: client connection to 127.21.27.130:45135: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:43:39.766917 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24452
I20260809 22:43:39.783174 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24595
W20260809 22:43:39.786018 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24595: Not authorized: Client connection negotiation failed: client connection to 127.21.27.131:34153: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:43:39.786121 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24595
I20260809 22:43:39.801798 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24204
I20260809 22:43:39.803022 24272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:39.908507 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24204
2026-08-09T22:43:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3705 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:40 dist-test-slave-chsk krb5kdc[24749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:40 dist-test-slave-chsk krb5kdc[24749](info): set up 2 sockets
Aug 09 22:43:40 dist-test-slave-chsk krb5kdc[24749](info): commencing operation
krb5kdc: starting...
W20260809 22:43:41.063513 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:41 dist-test-slave-chsk krb5kdc[24749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:41.286108 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38375
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:38375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:41.620169 24762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:41.620508 24762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:41.620678 24762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:41.629151 24762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:41.629317 24762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:41.629400 24762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:41.629482 24762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:41.629565 24762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:41.639972 24762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:38375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:41.641360 24762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:41.642731 24762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:41.655360 24770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:41.655391 24772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:41.655391 24769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:41.656451 24762 server_base.cc:1034] running on GCE node
I20260809 22:43:41.657333 24762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:41.658877 24762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:41.660122 24762 hybrid_clock.cc:648] HybridClock initialized: now 1786315421660055 us; error 66 us; skew 500 ppm
I20260809 22:43:41.660511 24762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:41 dist-test-slave-chsk krb5kdc[24749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315421, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:41.668246 24762 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:41.669888 24762 webserver.cc:460] Webserver started at http://127.21.27.190:32895/ using document root <none> and password file <none>
I20260809 22:43:41.670620 24762 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:41.670778 24762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:41.671115 24762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:41.674225 24762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "942f3785a64341eea60d6a452a13828c"
format_stamp: "Formatted at 2026-08-09 22:43:41 on dist-test-slave-chsk"
server_key: "48ef9c9ba052cb20a57c4145c5a74e74"
server_key_iv: "18aabcde521f883aba5a1cbd831474b1"
server_key_version: "encryptionkey@0"
I20260809 22:43:41.675065 24762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "942f3785a64341eea60d6a452a13828c"
format_stamp: "Formatted at 2026-08-09 22:43:41 on dist-test-slave-chsk"
server_key: "48ef9c9ba052cb20a57c4145c5a74e74"
server_key_iv: "18aabcde521f883aba5a1cbd831474b1"
server_key_version: "encryptionkey@0"
I20260809 22:43:41.680524 24762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:43:41.684173 24782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:41.685482 24762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:41.685674 24762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "942f3785a64341eea60d6a452a13828c"
format_stamp: "Formatted at 2026-08-09 22:43:41 on dist-test-slave-chsk"
server_key: "48ef9c9ba052cb20a57c4145c5a74e74"
server_key_iv: "18aabcde521f883aba5a1cbd831474b1"
server_key_version: "encryptionkey@0"
I20260809 22:43:41.685882 24762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:41.724783 24762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:41.730372 24762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:41.730717 24762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:41.754002 24762 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:38375
I20260809 22:43:41.754112 24834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:38375 every 8 connection(s)
I20260809 22:43:41.755628 24762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:41.758646 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24762
I20260809 22:43:41.758889 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:41.759794 21614 external_mini_cluster.cc:1428] Setting key 62c5b6b18a78e10a8f566b6fef8d645e
I20260809 22:43:41.760022 24835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:43:41 dist-test-slave-chsk krb5kdc[24749](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:41.775450 24835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c: Bootstrap starting.
I20260809 22:43:41.779247 24835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:41.780491 24835 log.cc:824] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:41.783470 24835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c: No bootstrap required, opened a new log
I20260809 22:43:41.784353 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:41.761717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.21.27.190:38375)
0809 22:43:41.762774 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:41.762793 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:41.763050 (+ 257us) server_negotiation.cc:407] Connection header received
0809 22:43:41.765080 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:41.765104 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:41.765474 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:41.765800 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:41.767879 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:41.769137 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:41.770875 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:41.771628 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:41.771785 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:41.776705 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:41.776713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:41.776727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:41.776766 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:41.780180 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:41.781016 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:41.781023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:41.781028 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:41.781193 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:41.781620 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:41.781625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:41.781628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:41.782170 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:41.782477 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:41.782920 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 22:43:41.783348 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":452,"threads_started":1}
I20260809 22:43:41.788208 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:43:41.791970 24835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } }
I20260809 22:43:41.792351 24835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:41.792460 24835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942f3785a64341eea60d6a452a13828c, State: Initialized, Role: FOLLOWER
I20260809 22:43:41.793015 24835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } }
I20260809 22:43:41.793262 24835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:41.793408 24835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:41.793582 24835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:41.795852 24835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } }
I20260809 22:43:41.796526 24835 leader_election.cc:302] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 942f3785a64341eea60d6a452a13828c; no voters:
I20260809 22:43:41.797206 24835 leader_election.cc:288] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:41.797291 24841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:41.798666 24841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [term 1 LEADER]: Becoming Leader. State: Replica: 942f3785a64341eea60d6a452a13828c, State: Running, Role: LEADER
I20260809 22:43:41.799329 24841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } }
I20260809 22:43:41.799674 24835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:41.805529 24843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 942f3785a64341eea60d6a452a13828c. Latest consensus state: current_term: 1 leader_uuid: "942f3785a64341eea60d6a452a13828c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } } }
I20260809 22:43:41.805624 24842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "942f3785a64341eea60d6a452a13828c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942f3785a64341eea60d6a452a13828c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38375 } } }
I20260809 22:43:41.805975 24843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:41.806062 24842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:41.807802 24851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:41.823753 24851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:41.834468 24851 catalog_manager.cc:1347] Generated new cluster ID: 8f721802638c4c2ca832aebfef6ced55
I20260809 22:43:41.834594 24851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:41.846592 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:41.838944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.21.27.190:38375)
0809 22:43:41.839126 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:41.839132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:41.839731 (+ 599us) server_negotiation.cc:407] Connection header received
0809 22:43:41.839948 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:41.839954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:41.840031 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:41.840154 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:41.841210 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:41.841954 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:41.843318 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:41.843740 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:41.843885 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:41.844985 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:41.844991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:41.844996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:41.845714 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:41.845759 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:41.845787 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:41.845928 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:41.846114 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 22:43:41.846206 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 22:43:41.848251 24851 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:41.850257 24851 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:41.860991 24851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 942f3785a64341eea60d6a452a13828c: Generated new TSK 0
I20260809 22:43:41.862124 24851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:43:41.877310 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24762
I20260809 22:43:41.878535 24830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:41.969972 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24762
2026-08-09T22:43:41Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2055 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:42 dist-test-slave-chsk krb5kdc[24869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:42 dist-test-slave-chsk krb5kdc[24869](info): set up 2 sockets
Aug 09 22:43:42 dist-test-slave-chsk krb5kdc[24869](info): commencing operation
krb5kdc: starting...
W20260809 22:43:43.103674 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:43 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:43.334460 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44009
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:44009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:43.677073 24882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:43.677402 24882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:43.677580 24882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:43.684940 24882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:43.685060 24882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:43.685154 24882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:43.685263 24882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:43.685350 24882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:43.695483 24882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:44009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:43.696892 24882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:43.698179 24882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:43.710098 24890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:43.710145 24888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:43.710263 24887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:43.710978 24882 server_base.cc:1034] running on GCE node
I20260809 22:43:43.711794 24882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:43.713145 24882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:43.714358 24882 hybrid_clock.cc:648] HybridClock initialized: now 1786315423714323 us; error 30 us; skew 500 ppm
I20260809 22:43:43.714764 24882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:43 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:43.718109 24882 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:43.719724 24882 webserver.cc:460] Webserver started at http://127.21.27.190:39739/ using document root <none> and password file <none>
I20260809 22:43:43.720427 24882 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:43.720567 24882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:43.720896 24882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:43.724117 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0b5d854117b446148ad6da7a94426682"
format_stamp: "Formatted at 2026-08-09 22:43:43 on dist-test-slave-chsk"
server_key: "058425dc878fb931a28d41a2e0c5948a"
server_key_iv: "298b88a5cfceb1883de2f227ef084f5e"
server_key_version: "encryptionkey@0"
I20260809 22:43:43.724941 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0b5d854117b446148ad6da7a94426682"
format_stamp: "Formatted at 2026-08-09 22:43:43 on dist-test-slave-chsk"
server_key: "058425dc878fb931a28d41a2e0c5948a"
server_key_iv: "298b88a5cfceb1883de2f227ef084f5e"
server_key_version: "encryptionkey@0"
I20260809 22:43:43.731607 24882 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:43:43.736645 24898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:43.739461 24882 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:43.739673 24882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0b5d854117b446148ad6da7a94426682"
format_stamp: "Formatted at 2026-08-09 22:43:43 on dist-test-slave-chsk"
server_key: "058425dc878fb931a28d41a2e0c5948a"
server_key_iv: "298b88a5cfceb1883de2f227ef084f5e"
server_key_version: "encryptionkey@0"
I20260809 22:43:43.739885 24882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:43.759109 24882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:43.763803 24882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:43.764146 24882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:43.783371 24882 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:44009
I20260809 22:43:43.783394 24956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:44009 every 8 connection(s)
I20260809 22:43:43.784847 24882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:43.787763 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24882
I20260809 22:43:43.787977 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:43.788679 21614 external_mini_cluster.cc:1428] Setting key 2fae0ff6ada5931b88a76b88caefbea0
I20260809 22:43:43.788879 24957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:43.803527 24957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682: Bootstrap starting.
Aug 09 22:43:43 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:43.807606 24957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:43.809042 24957 log.cc:824] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:43.812979 24957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682: No bootstrap required, opened a new log
I20260809 22:43:43.814432 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:43.790771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.21.27.190:44009)
0809 22:43:43.791657 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:43.791674 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:43.791733 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:43:43.793924 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:43.793955 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:43.794493 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:43.794964 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:43.796327 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:43.798313 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:43.799686 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:43.800817 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:43.801016 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:43.805154 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:43.805165 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:43.805186 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:43.805234 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:43.809421 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:43.810424 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:43.810432 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:43.810438 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:43.810711 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:43.811117 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:43.811122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:43.811126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:43.811894 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:43.812217 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:43.812597 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 22:43:43.813058 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":368,"threads_started":1}
I20260809 22:43:43.821887 24957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } }
I20260809 22:43:43.822273 24957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:43.822361 24957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b5d854117b446148ad6da7a94426682, State: Initialized, Role: FOLLOWER
I20260809 22:43:43.822850 24957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } }
I20260809 22:43:43.823057 24957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:43.823210 24957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:43.823416 24957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:43.825951 24957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } }
I20260809 22:43:43.826470 24957 leader_election.cc:302] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b5d854117b446148ad6da7a94426682; no voters:
I20260809 22:43:43.827122 24957 leader_election.cc:288] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:43.827421 24962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:43.828145 24962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [term 1 LEADER]: Becoming Leader. State: Replica: 0b5d854117b446148ad6da7a94426682, State: Running, Role: LEADER
I20260809 22:43:43.828787 24962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } }
I20260809 22:43:43.829486 24957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:43.832163 24964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b5d854117b446148ad6da7a94426682. Latest consensus state: current_term: 1 leader_uuid: "0b5d854117b446148ad6da7a94426682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } } }
I20260809 22:43:43.832139 24963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b5d854117b446148ad6da7a94426682" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b5d854117b446148ad6da7a94426682" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44009 } } }
I20260809 22:43:43.832561 24964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:43.832616 24963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:43.834115 24971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:43.842988 24971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:43.852262 24971 catalog_manager.cc:1347] Generated new cluster ID: fbcda7f01a444023bb1e68064a13adaf
I20260809 22:43:43.852392 24971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:43.863701 24971 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:43.865670 24971 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:43.883044 24971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b5d854117b446148ad6da7a94426682: Generated new TSK 0
I20260809 22:43:43.884256 24971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:43.938903 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:44.283046 24984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:44.283438 24984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:44.283643 24984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:44.290987 24984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:44.291108 24984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:44.291360 24984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:44.301409 24984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:44.302834 24984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:44.304278 24984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:44.319216 24990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:44.319308 24989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:44.319341 24992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:44.320083 24984 server_base.cc:1034] running on GCE node
I20260809 22:43:44.320773 24984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:44.321713 24984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:44.322918 24984 hybrid_clock.cc:648] HybridClock initialized: now 1786315424322883 us; error 33 us; skew 500 ppm
I20260809 22:43:44.323311 24984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:44 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:44.326578 24984 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:44.328191 24984 webserver.cc:460] Webserver started at http://127.21.27.129:43273/ using document root <none> and password file <none>
I20260809 22:43:44.328890 24984 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:44.329027 24984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:44.329385 24984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:44.332474 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "b4e6b728ba554e5eb4d8cc27f6074f63"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "562a319f8ab2040e12e1be842f01140e"
server_key_iv: "4ac2861f87f19c4173a9327c71e68806"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.333284 24984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "b4e6b728ba554e5eb4d8cc27f6074f63"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "562a319f8ab2040e12e1be842f01140e"
server_key_iv: "4ac2861f87f19c4173a9327c71e68806"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.338502 24984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:44.342319 25000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.343859 24984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:43:44.344061 24984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "b4e6b728ba554e5eb4d8cc27f6074f63"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "562a319f8ab2040e12e1be842f01140e"
server_key_iv: "4ac2861f87f19c4173a9327c71e68806"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.344353 24984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:44.397523 24984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:44.401894 24984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:44.402184 24984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:44.403128 24984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:44.404594 24984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:44.404706 24984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.404858 24984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:44.404933 24984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.439507 24984 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:35835
I20260809 22:43:44.439579 25119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:35835 every 8 connection(s)
I20260809 22:43:44.440905 24984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:44.442889 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 24984
I20260809 22:43:44.443163 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:44.444096 21614 external_mini_cluster.cc:1428] Setting key 7c001bb5a0982e2438cb94ae052b3e24
Aug 09 22:43:44 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:44.464583 25122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:44.443964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:40987 (local address 127.21.27.190:44009)
0809 22:43:44.449332 (+ 5368us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:44.449337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:44.449354 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:43:44.449425 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:44.449429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:44.449539 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:44.449664 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:44.451363 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.452588 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.454187 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.454934 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.455081 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:44.458738 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:44.458744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:44.458747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:44.458781 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:44.461430 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:44.462200 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:44.462205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:44.462207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:44.462324 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:44.462771 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:44.462776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:44.462781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:44.463050 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:44.463240 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:44.463954 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 22:43:44.464187 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5261,"thread_start_us":5132,"threads_started":1}
I20260809 22:43:44.465544 25123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:44.444670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44009 (local address 127.21.27.129:40987)
0809 22:43:44.445562 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:44.445608 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:44.447345 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:44.449887 (+ 2542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:44.449896 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:44.450550 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:44.451124 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:44.451198 (+ 74us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.452760 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.452764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:44.454020 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:44.454032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.455198 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.455213 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:44.455431 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:44.456164 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:44.456187 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:44.458500 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:44.461571 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:44.461582 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:44.461599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:44.462056 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:44.462438 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:44.462443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:44.462447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:44.462651 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:44.463232 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:44.463241 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:44.463572 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 22:43:44.463837 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 22:43:44.464399 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":359,"threads_started":1}
I20260809 22:43:44.467229 25120 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44009
I20260809 22:43:44.467592 25120 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:44.468580 25120 heartbeater.cc:511] Master 127.21.27.190:44009 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:44.471094 24917 ts_manager.cc:194] Registered new tserver with Master: b4e6b728ba554e5eb4d8cc27f6074f63 (127.21.27.129:35835)
I20260809 22:43:44.472682 24917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:40987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:44.498595 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:44.832391 25126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:44.832700 25126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:44.832798 25126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:44.840366 25126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:44.840487 25126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:44.840606 25126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:44.851436 25126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:44.852736 25126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:44.854015 25126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:44.867336 25134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:44.867376 25132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:44.867937 25131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:44.868630 25126 server_base.cc:1034] running on GCE node
I20260809 22:43:44.869433 25126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:44.870654 25126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:44.871927 25126 hybrid_clock.cc:648] HybridClock initialized: now 1786315424871846 us; error 59 us; skew 500 ppm
I20260809 22:43:44.872325 25126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:44 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:44.876145 25126 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:44.877609 25126 webserver.cc:460] Webserver started at http://127.21.27.130:32771/ using document root <none> and password file <none>
I20260809 22:43:44.878280 25126 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:44.878387 25126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:44.878647 25126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:44.881703 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "9e4331e5e6bb4187aeb242c437d1db39"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "eb8d5b6a433748c48c7426ce9a93caf8"
server_key_iv: "ce58e3ce3e8ae56cc1d391cb47448ce2"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.882484 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "9e4331e5e6bb4187aeb242c437d1db39"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "eb8d5b6a433748c48c7426ce9a93caf8"
server_key_iv: "ce58e3ce3e8ae56cc1d391cb47448ce2"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.887845 25126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:43:44.891508 25142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.892921 25126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:43:44.893098 25126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "9e4331e5e6bb4187aeb242c437d1db39"
format_stamp: "Formatted at 2026-08-09 22:43:44 on dist-test-slave-chsk"
server_key: "eb8d5b6a433748c48c7426ce9a93caf8"
server_key_iv: "ce58e3ce3e8ae56cc1d391cb47448ce2"
server_key_version: "encryptionkey@0"
I20260809 22:43:44.893301 25126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:44.918076 25126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:44.922495 25126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:44.922823 25126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:44.923938 25126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:44.925346 25126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:44.925423 25126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.925596 25126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:44.925659 25126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:44.960127 25126 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:33703
I20260809 22:43:44.960217 25258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:33703 every 8 connection(s)
I20260809 22:43:44.961506 25126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:44.961907 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25126
I20260809 22:43:44.962121 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:44.962778 21614 external_mini_cluster.cc:1428] Setting key c1a77140691d62eea65e0ce4b0b9e0d2
Aug 09 22:43:44 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:44.984889 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:44.965017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:56733 (local address 127.21.27.190:44009)
0809 22:43:44.969575 (+ 4558us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:44.969579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:44.969595 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:43:44.969651 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:44.969655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:44.969711 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:44.969838 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:44.971418 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.972127 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.973689 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.974134 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.974273 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:44.978274 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:44.978279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:44.978282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:44.978318 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:44.981162 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:44.982242 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:44.982246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:44.982249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:44.982396 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:44.982948 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:44.982951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:44.982953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:44.983292 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:44.983408 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:44.984416 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0809 22:43:44.984601 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4459,"thread_start_us":4367,"threads_started":1}
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
I20260809 22:43:44.986539 25261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:44.964811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44009 (local address 127.21.27.130:56733)
0809 22:43:44.967315 (+ 2504us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:44.967371 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:44.969042 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:44.970089 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:44.970097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:44.970616 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:44.971240 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:44.971260 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.972291 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.972303 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:44.973530 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:44.973541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:44.974381 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:44.974388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:44.974623 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:44.975373 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:44.975396 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:44.978013 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:44.981322 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:44.981334 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:44.981356 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:44.982107 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:44.982520 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:44.982528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:44.982535 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:44.982826 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:44.983437 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:44.983449 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:44.983968 (+ 519us) client_negotiation.cc:769] Sending connection context
0809 22:43:44.984328 (+ 360us) client_negotiation.cc:240] Negotiation successful
0809 22:43:44.984977 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2188,"thread_start_us":1958,"threads_started":1}
I20260809 22:43:44.988175 25259 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44009
I20260809 22:43:44.988583 25259 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
I20260809 22:43:44.989449 25259 heartbeater.cc:511] Master 127.21.27.190:44009 requested a full tablet report, sending...
I20260809 22:43:44.991017 24917 ts_manager.cc:194] Registered new tserver with Master: 9e4331e5e6bb4187aeb242c437d1db39 (127.21.27.130:33703)
I20260809 22:43:44.991907 24917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:56733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:45.016165 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:45.347015 25266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:45.347391 25266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:45.347587 25266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:45.355077 25266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:45.355279 25266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:45.355499 25266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:45.365759 25266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:45.367122 25266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:45.368538 25266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:45.382107 25271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:45.382169 25272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:45.382366 25274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:45.383275 25266 server_base.cc:1034] running on GCE node
I20260809 22:43:45.383898 25266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:45.384855 25266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:45.386111 25266 hybrid_clock.cc:648] HybridClock initialized: now 1786315425386061 us; error 62 us; skew 500 ppm
I20260809 22:43:45.386442 25266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:45.389740 25266 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:45.391309 25266 webserver.cc:460] Webserver started at http://127.21.27.131:42983/ using document root <none> and password file <none>
I20260809 22:43:45.391961 25266 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:45.392071 25266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:45.392323 25266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:45.395217 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "d78ae34da11c45438847484682b663ed"
format_stamp: "Formatted at 2026-08-09 22:43:45 on dist-test-slave-chsk"
server_key: "9753a141691b8b4859ace29d429da598"
server_key_iv: "0aa619a046c609583a37a856332aa30a"
server_key_version: "encryptionkey@0"
I20260809 22:43:45.395990 25266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "d78ae34da11c45438847484682b663ed"
format_stamp: "Formatted at 2026-08-09 22:43:45 on dist-test-slave-chsk"
server_key: "9753a141691b8b4859ace29d429da598"
server_key_iv: "0aa619a046c609583a37a856332aa30a"
server_key_version: "encryptionkey@0"
I20260809 22:43:45.401428 25266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 22:43:45.404948 25282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:45.406309 25266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:43:45.406492 25266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "d78ae34da11c45438847484682b663ed"
format_stamp: "Formatted at 2026-08-09 22:43:45 on dist-test-slave-chsk"
server_key: "9753a141691b8b4859ace29d429da598"
server_key_iv: "0aa619a046c609583a37a856332aa30a"
server_key_version: "encryptionkey@0"
I20260809 22:43:45.406736 25266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:45.433758 25266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:45.438023 25266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:45.438359 25266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:45.439503 25266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:45.440922 25266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:45.440994 25266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:45.441154 25266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:45.441207 25266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:45.475790 25266 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:38773
I20260809 22:43:45.475837 25120 heartbeater.cc:503] Master 127.21.27.190:44009 was elected leader, sending a full tablet report...
I20260809 22:43:45.475803 25402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:38773 every 8 connection(s)
I20260809 22:43:45.477356 25266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:45.478654 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25266
I20260809 22:43:45.478848 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:45.479528 21614 external_mini_cluster.cc:1428] Setting key bd798b6b4331a1627386c8b768b78fb2
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:45.497879 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.480445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:37235 (local address 127.21.27.190:44009)
0809 22:43:45.480643 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:45.480651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:45.481865 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 22:43:45.483751 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:45.483755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:45.483820 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:45.483903 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:45.485327 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.486071 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.487622 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.488008 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.488129 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.491519 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:45.491523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:45.491526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:45.491560 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:45.494645 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.495473 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.495477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.495479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.495601 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.496044 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.496055 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.496058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.496340 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:45.496525 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:45.497298 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 22:43:45.497538 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:43:45.498931 25406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.480877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44009 (local address 127.21.27.131:37235)
0809 22:43:45.481695 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:45.481749 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:45.483502 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:45.484109 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:45.484116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:45.484614 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:45.485175 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:45.485193 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.486193 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.486196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:45.487488 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:45.487506 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.488249 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.488253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:45.488448 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.489071 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:45.489092 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:45.491335 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:45.494788 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:45.494798 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:45.494821 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:45.495339 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:45.495717 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:45.495722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:45.495726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:45.495918 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:45.496500 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:45.496508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:45.496858 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 22:43:45.497202 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 22:43:45.497763 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":359,"threads_started":1}
I20260809 22:43:45.500432 25403 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44009
I20260809 22:43:45.500741 25403 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:45.501446 25403 heartbeater.cc:511] Master 127.21.27.190:44009 requested a full tablet report, sending...
I20260809 22:43:45.502898 24917 ts_manager.cc:194] Registered new tserver with Master: d78ae34da11c45438847484682b663ed (127.21.27.131:38773)
I20260809 22:43:45.503768 24917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:37235
I20260809 22:43:45.507508 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:45.562050 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.547571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.21.27.190:44009)
0809 22:43:45.547780 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:45.547787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:45.548203 (+ 416us) server_negotiation.cc:407] Connection header received
0809 22:43:45.548437 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:45.548443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:45.548523 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:45.548656 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:45.550085 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.550821 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.552157 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.552590 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.552731 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.556700 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:45.556706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:45.556709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:45.556745 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:45.559252 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.559932 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.559939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.559942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.560097 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.560494 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.560499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.560502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.560847 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:45.561126 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:45.561460 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:43:45.561694 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 22:43:45.562651 24917 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49572
I20260809 22:43:45.564555 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24984
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:45.579983 25415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.565549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.21.27.129:35835)
0809 22:43:45.566283 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:45.566291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:45.566317 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:43:45.566371 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:45.566377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:45.566569 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:45.566700 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:45.567877 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.568948 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.570155 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.570693 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.570791 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.574675 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:45.574686 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:45.574699 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:45.574731 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:45.577467 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.578072 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.578078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.578081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.578201 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.578598 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.578601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.578603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.578929 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:45.579238 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:45.579506 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 22:43:45.579709 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":381,"threads_started":1}
W20260809 22:43:45.581210 25115 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57628
W20260809 22:43:45.582336 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24984: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:45.582557 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24984
I20260809 22:43:45.600912 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25126
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:45.618216 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.601812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.21.27.130:33703)
0809 22:43:45.602482 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:45.602489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:45.602518 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:43:45.602581 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:45.602588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:45.602799 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:45.602934 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:45.604383 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.605464 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.606799 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.607492 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.607599 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.612269 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:45.612281 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:45.612296 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:45.612344 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:45.615629 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.616284 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.616291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.616293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.616417 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.616746 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.616749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.616751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.617086 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:45.617315 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:45.617586 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 22:43:45.617838 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":365,"threads_started":1}
W20260809 22:43:45.619697 25253 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33982
W20260809 22:43:45.620841 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25126: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:45.620998 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25126
I20260809 22:43:45.639103 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25266
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315425, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:45.655916 25418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:45.640387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.21.27.131:38773)
0809 22:43:45.641105 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:45.641112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:45.641142 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:43:45.641203 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:45.641209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:45.641422 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:45.641549 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:45.642869 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.644081 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.645328 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:45.645883 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:45.645989 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:45.650253 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:45.650265 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:45.650279 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:45.650314 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:45.653337 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.653974 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.653981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.653983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.654103 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:45.654415 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:45.654418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:45.654420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:45.654770 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:45.655002 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:45.655287 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 22:43:45.655491 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":379,"threads_started":1}
W20260809 22:43:45.657256 25398 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36874
W20260809 22:43:45.658352 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25266: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:43:45.658558 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25266
I20260809 22:43:45.676738 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 24882
I20260809 22:43:45.677989 24951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:45.801034 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 24882
2026-08-09T22:43:45Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3836 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[25424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[25424](info): set up 2 sockets
Aug 09 22:43:45 dist-test-slave-chsk krb5kdc[25424](info): commencing operation
krb5kdc: starting...
W20260809 22:43:46.960466 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:47 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:47.178714 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34583
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:34583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:47.504094 25437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:47.504405 25437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:47.504494 25437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:47.512055 25437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:47.512187 25437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:47.512236 25437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:47.512297 25437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:47.512347 25437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:47.522563 25437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:34583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:47.524001 25437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:47.525274 25437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:47.537354 25442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:47.537388 25446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:47.537417 25444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:47.538261 25437 server_base.cc:1034] running on GCE node
I20260809 22:43:47.539067 25437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:47.540585 25437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:47.541837 25437 hybrid_clock.cc:648] HybridClock initialized: now 1786315427541799 us; error 49 us; skew 500 ppm
I20260809 22:43:47.542217 25437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:47 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315427, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:47.550505 25437 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:47.552132 25437 webserver.cc:460] Webserver started at http://127.21.27.190:36537/ using document root <none> and password file <none>
I20260809 22:43:47.552909 25437 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:47.553061 25437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:47.553392 25437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:47.556573 25437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "f7842b178fa84c4296e7696a2e1e7988"
format_stamp: "Formatted at 2026-08-09 22:43:47 on dist-test-slave-chsk"
server_key: "10ec0d8336feed5bb59ed15cf8ed981f"
server_key_iv: "8e5f8bbc6f46e4296e594de165772630"
server_key_version: "encryptionkey@0"
I20260809 22:43:47.557411 25437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "f7842b178fa84c4296e7696a2e1e7988"
format_stamp: "Formatted at 2026-08-09 22:43:47 on dist-test-slave-chsk"
server_key: "10ec0d8336feed5bb59ed15cf8ed981f"
server_key_iv: "8e5f8bbc6f46e4296e594de165772630"
server_key_version: "encryptionkey@0"
I20260809 22:43:47.562594 25437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:43:47.565982 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:47.567250 25437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:43:47.567428 25437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "f7842b178fa84c4296e7696a2e1e7988"
format_stamp: "Formatted at 2026-08-09 22:43:47 on dist-test-slave-chsk"
server_key: "10ec0d8336feed5bb59ed15cf8ed981f"
server_key_iv: "8e5f8bbc6f46e4296e594de165772630"
server_key_version: "encryptionkey@0"
I20260809 22:43:47.567617 25437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:47.586948 25437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:47.590864 25437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:47.591472 25437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:47.610772 25437 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:34583
I20260809 22:43:47.610790 25508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:34583 every 8 connection(s)
I20260809 22:43:47.612221 25437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:47.615954 25509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:47.619592 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25437
I20260809 22:43:47.619827 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:47.620478 21614 external_mini_cluster.cc:1428] Setting key 3ac627a91cd4c7719fb4fb76d2c7b235
I20260809 22:43:47.630344 25509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988: Bootstrap starting.
I20260809 22:43:47.634377 25509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:47.635744 25509 log.cc:824] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:47.638767 25509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988: No bootstrap required, opened a new log
Aug 09 22:43:47 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:47.646816 25509 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } }
I20260809 22:43:47.647217 25509 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:47.647361 25509 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7842b178fa84c4296e7696a2e1e7988, State: Initialized, Role: FOLLOWER
I20260809 22:43:47.647997 25509 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } }
I20260809 22:43:47.648281 25509 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:47.648439 25509 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:47.648629 25509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:47.650919 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:47.622609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.21.27.190:34583)
0809 22:43:47.629299 (+ 6690us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:47.629316 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:47.629368 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:43:47.632000 (+ 2632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:47.632030 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:47.632601 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:47.633062 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:47.634457 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:47.636507 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:47.637955 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:47.638956 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:47.639159 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:47.643301 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:47.643309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:47.643323 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:47.643365 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:47.646540 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:47.647336 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:47.647352 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:47.647360 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:47.647583 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:47.648001 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:47.648006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:47.648010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:47.648710 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:47.649013 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:47.649545 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 22:43:47.649965 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6451,"thread_start_us":6224,"threads_started":1}
I20260809 22:43:47.651201 25509 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } }
I20260809 22:43:47.651773 25509 leader_election.cc:302] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7842b178fa84c4296e7696a2e1e7988; no voters:
I20260809 22:43:47.652444 25509 leader_election.cc:288] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:47.652565 25515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:47.653836 25515 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [term 1 LEADER]: Becoming Leader. State: Replica: f7842b178fa84c4296e7696a2e1e7988, State: Running, Role: LEADER
I20260809 22:43:47.654458 25515 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } }
I20260809 22:43:47.654860 25509 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:47.657543 25517 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7842b178fa84c4296e7696a2e1e7988. Latest consensus state: current_term: 1 leader_uuid: "f7842b178fa84c4296e7696a2e1e7988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } } }
I20260809 22:43:47.658094 25517 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:47.657752 25516 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7842b178fa84c4296e7696a2e1e7988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7842b178fa84c4296e7696a2e1e7988" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34583 } } }
I20260809 22:43:47.658347 25516 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:47.664430 25524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:47.669766 25524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:47.678167 25524 catalog_manager.cc:1347] Generated new cluster ID: a13014d529d04a0f9f543231b9063888
I20260809 22:43:47.678287 25524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:47.701128 25524 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:47.703068 25524 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:47.712709 25524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7842b178fa84c4296e7696a2e1e7988: Generated new TSK 0
I20260809 22:43:47.713843 25524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:47.782294 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:48.104151 25538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:48.104461 25538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:48.104629 25538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:48.111938 25538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:48.112085 25538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:48.112258 25538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:48.121968 25538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:48.123329 25538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:48.124716 25538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:48.137861 25543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:48.137825 25546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:48.137791 25544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:48.138660 25538 server_base.cc:1034] running on GCE node
I20260809 22:43:48.139354 25538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:48.140285 25538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:48.141518 25538 hybrid_clock.cc:648] HybridClock initialized: now 1786315428141476 us; error 63 us; skew 500 ppm
I20260809 22:43:48.141830 25538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:48 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315428, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:48.145121 25538 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:48.146549 25538 webserver.cc:460] Webserver started at http://127.21.27.129:36191/ using document root <none> and password file <none>
I20260809 22:43:48.147322 25538 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:48.147475 25538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:48.147797 25538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:48.150707 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "7b7d5a09b4eb4ba1909fe8a911117850"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "e26c7d85c352cbbc2da8b9e46611aa22"
server_key_iv: "ac3471bef26cc1b33ff31619e0492ff8"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.151558 25538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "7b7d5a09b4eb4ba1909fe8a911117850"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "e26c7d85c352cbbc2da8b9e46611aa22"
server_key_iv: "ac3471bef26cc1b33ff31619e0492ff8"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.156854 25538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 22:43:48.160326 25554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.161628 25538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:43:48.161809 25538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "7b7d5a09b4eb4ba1909fe8a911117850"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "e26c7d85c352cbbc2da8b9e46611aa22"
server_key_iv: "ac3471bef26cc1b33ff31619e0492ff8"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.162040 25538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:48.177423 25538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:48.181871 25538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:48.182219 25538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:48.183351 25538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:48.184760 25538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:48.184865 25538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.185019 25538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:48.185096 25538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.219385 25538 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:42237
I20260809 22:43:48.219467 25678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:42237 every 8 connection(s)
I20260809 22:43:48.220715 25538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:48.223944 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25538
I20260809 22:43:48.224182 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:48.224860 21614 external_mini_cluster.cc:1428] Setting key c84657afe978e196078293ce4c3b8008
Aug 09 22:43:48 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315428, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:48.244781 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:48.223436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48055 (local address 127.21.27.190:34583)
0809 22:43:48.223980 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:48.223987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:48.225241 (+ 1254us) server_negotiation.cc:407] Connection header received
0809 22:43:48.227454 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:48.227460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:48.227553 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:48.227676 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:48.229143 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.230296 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.231923 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.232686 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.232833 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:48.236579 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:48.236588 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:48.236596 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:48.236644 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:48.240351 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:48.241322 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:48.241330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:48.241336 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:48.241514 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:48.242130 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:48.242135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:48.242138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:48.242578 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:48.242773 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:48.243913 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0809 22:43:48.244300 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":270,"threads_started":1}
I20260809 22:43:48.245618 25682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:48.223959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34583 (local address 127.21.27.129:48055)
0809 22:43:48.225051 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:48.225098 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:48.227120 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:48.227878 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:48.227886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:48.228410 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:48.228961 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:48.228977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.230462 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.230465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:48.231774 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:48.231787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.232903 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.232908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:48.233104 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:48.233817 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:48.233839 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:48.236340 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:48.240500 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:48.240507 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:48.240521 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:48.241151 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:48.241660 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:48.241667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:48.241673 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:48.241939 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:48.242857 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:48.242865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:48.243400 (+ 535us) client_negotiation.cc:769] Sending connection context
0809 22:43:48.243730 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 22:43:48.244335 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":551,"threads_started":1}
I20260809 22:43:48.247344 25679 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34583
I20260809 22:43:48.247704 25679 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:48.248677 25679 heartbeater.cc:511] Master 127.21.27.190:34583 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:48.250981 25472 ts_manager.cc:194] Registered new tserver with Master: 7b7d5a09b4eb4ba1909fe8a911117850 (127.21.27.129:42237)
I20260809 22:43:48.252435 25472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:48055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:48.278478 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:48.602567 25685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:48.602850 25685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:48.602946 25685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:48.610389 25685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:48.610527 25685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:48.610708 25685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:48.626252 25685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:48.627784 25685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:48.629120 25685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:48.642283 25690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:48.642241 25691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:48.642310 25693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:48.643493 25685 server_base.cc:1034] running on GCE node
I20260809 22:43:48.644153 25685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:48.645109 25685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:48.646313 25685 hybrid_clock.cc:648] HybridClock initialized: now 1786315428646277 us; error 26 us; skew 500 ppm
I20260809 22:43:48.646642 25685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:48 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315428, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:48.649788 25685 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:48.651304 25685 webserver.cc:460] Webserver started at http://127.21.27.130:43481/ using document root <none> and password file <none>
I20260809 22:43:48.651978 25685 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:48.652132 25685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:48.652449 25685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:48.655424 25685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "ff42a575631b4544a4de9fb1de6f6d55"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "7ecf65260022533e77fc60a3b4305475"
server_key_iv: "b7d622e270344fd82d6220fa4414c9f3"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.656217 25685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "ff42a575631b4544a4de9fb1de6f6d55"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "7ecf65260022533e77fc60a3b4305475"
server_key_iv: "b7d622e270344fd82d6220fa4414c9f3"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.661249 25685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:48.664866 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.666155 25685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:43:48.666318 25685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "ff42a575631b4544a4de9fb1de6f6d55"
format_stamp: "Formatted at 2026-08-09 22:43:48 on dist-test-slave-chsk"
server_key: "7ecf65260022533e77fc60a3b4305475"
server_key_iv: "b7d622e270344fd82d6220fa4414c9f3"
server_key_version: "encryptionkey@0"
I20260809 22:43:48.666538 25685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:48.681049 25685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:48.685128 25685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:48.685386 25685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:48.686270 25685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:48.687599 25685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:48.687673 25685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.687757 25685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:48.687809 25685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:48.720038 25685 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:35693
I20260809 22:43:48.720079 25825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:35693 every 8 connection(s)
I20260809 22:43:48.721398 25685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:48.729943 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25685
I20260809 22:43:48.730158 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:48.730595 21614 external_mini_cluster.cc:1428] Setting key 54e54f0c2a0879145dd64a899e1a7e5f
Aug 09 22:43:48 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315428, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:48.742628 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:48.724091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:40305 (local address 127.21.27.190:34583)
0809 22:43:48.724291 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:48.724296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:48.725520 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 22:43:48.727410 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:48.727415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:48.727511 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:48.727624 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:48.729153 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.729861 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.731684 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.732140 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.732304 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:48.736451 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:48.736457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:48.736460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:48.736494 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:48.739127 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:48.740102 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:48.740110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:48.740115 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:48.740294 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:48.740728 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:48.740736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:48.740741 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:48.741157 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:48.741289 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:48.742201 (+ 912us) server_negotiation.cc:299] Negotiation successful
0809 22:43:48.742381 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:43:48.744251 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:48.724553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34583 (local address 127.21.27.130:40305)
0809 22:43:48.725369 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:48.725406 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:48.727081 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:48.727848 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:48.727855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:48.728432 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:48.728981 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:48.728997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.730024 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.730029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:48.731532 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:48.731544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:48.732419 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:48.732428 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:48.732717 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:48.733436 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:48.733458 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:48.736181 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:48.739341 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:48.739353 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:48.739373 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:48.739940 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:48.740410 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:48.740413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:48.740415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:48.740596 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:48.741299 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:48.741310 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:48.741785 (+ 475us) client_negotiation.cc:769] Sending connection context
0809 22:43:48.742101 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 22:43:48.742710 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":309,"threads_started":1}
I20260809 22:43:48.745787 25826 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34583
I20260809 22:43:48.746227 25826 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:48.746965 25826 heartbeater.cc:511] Master 127.21.27.190:34583 requested a full tablet report, sending...
I20260809 22:43:48.748476 25472 ts_manager.cc:194] Registered new tserver with Master: ff42a575631b4544a4de9fb1de6f6d55 (127.21.27.130:35693)
I20260809 22:43:48.749269 25472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:40305
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:48.780560 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:49.105772 25831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:49.106057 25831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:49.106151 25831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:49.113224 25831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:49.113336 25831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:49.113449 25831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:49.122886 25831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:49.124176 25831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:49.125406 25831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:49.138187 25837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:49.138224 25836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:49.138407 25839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:49.139387 25831 server_base.cc:1034] running on GCE node
I20260809 22:43:49.140020 25831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:49.140911 25831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:49.142194 25831 hybrid_clock.cc:648] HybridClock initialized: now 1786315429142150 us; error 64 us; skew 500 ppm
I20260809 22:43:49.142532 25831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:49.145876 25831 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:49.147413 25831 webserver.cc:460] Webserver started at http://127.21.27.131:36433/ using document root <none> and password file <none>
I20260809 22:43:49.148097 25831 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:49.148242 25831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:49.148554 25831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:49.151526 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "e63e2205ac0f4ed89b1a0b125fdcbebc"
format_stamp: "Formatted at 2026-08-09 22:43:49 on dist-test-slave-chsk"
server_key: "b005527145e63df7e0fe6d87a439d838"
server_key_iv: "57eb20a8e253c20a403079a606ffdffe"
server_key_version: "encryptionkey@0"
I20260809 22:43:49.152325 25831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "e63e2205ac0f4ed89b1a0b125fdcbebc"
format_stamp: "Formatted at 2026-08-09 22:43:49 on dist-test-slave-chsk"
server_key: "b005527145e63df7e0fe6d87a439d838"
server_key_iv: "57eb20a8e253c20a403079a606ffdffe"
server_key_version: "encryptionkey@0"
I20260809 22:43:49.157411 25831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:49.160810 25847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:49.162071 25831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:43:49.162253 25831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "e63e2205ac0f4ed89b1a0b125fdcbebc"
format_stamp: "Formatted at 2026-08-09 22:43:49 on dist-test-slave-chsk"
server_key: "b005527145e63df7e0fe6d87a439d838"
server_key_iv: "57eb20a8e253c20a403079a606ffdffe"
server_key_version: "encryptionkey@0"
I20260809 22:43:49.162446 25831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:49.179877 25831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:49.184015 25831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:49.184310 25831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:49.185261 25831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:49.186611 25831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:49.186685 25831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:49.186862 25831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:49.186913 25831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:49.220434 25831 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:41695
I20260809 22:43:49.220458 25970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:41695 every 8 connection(s)
I20260809 22:43:49.221727 25831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:49.231603 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 25831
I20260809 22:43:49.231828 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:49.232367 21614 external_mini_cluster.cc:1428] Setting key 9a2f785b6fcc17ddcad447ad8e13f212
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:49.246732 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.224704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:52211 (local address 127.21.27.190:34583)
0809 22:43:49.224910 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.224915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.225976 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 22:43:49.227979 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.227985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.228106 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.228230 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.229737 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.230496 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.232617 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.233137 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.233274 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.237224 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.237229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.237233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.237267 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.241394 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.242535 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.242540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.242544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.242744 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.243379 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.243384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.243387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.243822 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.244010 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.245075 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.245359 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:43:49.247905 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.225104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34583 (local address 127.21.27.131:52211)
0809 22:43:49.225817 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:49.225855 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:49.227703 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:49.228472 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:49.228479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:49.229002 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:49.229552 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:49.229570 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.230664 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.230668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:49.232459 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:49.232478 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.233392 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.233399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:49.233656 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.234492 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:49.234526 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:49.236994 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:49.241578 (+ 4584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:49.241588 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:49.241625 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:49.242383 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:49.242899 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:49.242906 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:49.242911 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:49.243235 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:49.243989 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:49.243998 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:49.244543 (+ 545us) client_negotiation.cc:769] Sending connection context
0809 22:43:49.244894 (+ 351us) client_negotiation.cc:240] Negotiation successful
0809 22:43:49.245478 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":281,"threads_started":1}
I20260809 22:43:49.250089 25971 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34583
I20260809 22:43:49.250496 25971 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:49.251438 25971 heartbeater.cc:511] Master 127.21.27.190:34583 requested a full tablet report, sending...
I20260809 22:43:49.252972 25472 ts_manager.cc:194] Registered new tserver with Master: e63e2205ac0f4ed89b1a0b125fdcbebc (127.21.27.131:41695)
I20260809 22:43:49.254248 25472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:52211
I20260809 22:43:49.255416 25679 heartbeater.cc:503] Master 127.21.27.190:34583 was elected leader, sending a full tablet report...
I20260809 22:43:49.263458 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:49.323990 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.308458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.21.27.129:42237)
0809 22:43:49.309148 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.309159 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.309193 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:43:49.309252 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.309258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.309466 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.309643 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.311083 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.312130 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.313385 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.313952 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.314097 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.318201 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.318213 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.318229 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.318262 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.321084 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.321766 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.321773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.321776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.321899 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.322302 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.322308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.322312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.322670 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.322992 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.323353 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.323642 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":308,"threads_started":1}
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:49.342105 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.328548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.21.27.190:34583)
0809 22:43:49.328769 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.328776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.328941 (+ 165us) server_negotiation.cc:407] Connection header received
0809 22:43:49.329137 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.329143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.329218 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.329322 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.330539 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.331441 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.333033 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.333454 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.333588 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.337387 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.337392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.337395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.337427 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.339938 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.340515 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.340518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.340521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.340636 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.340954 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.340957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.340958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.341275 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.341455 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.341710 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.341866 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 22:43:49.342677 25472 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41742
I20260809 22:43:49.343641 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25538
I20260809 22:43:49.356472 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.344516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.21.27.129:42237)
0809 22:43:49.344743 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.344748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.344976 (+ 228us) server_negotiation.cc:407] Connection header received
0809 22:43:49.345162 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.345171 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.345234 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.345338 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.346707 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.347443 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.348659 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.349056 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.349192 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.351408 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.351413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.351417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.351448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.353773 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.354426 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.354432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.354437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.354560 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.354966 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.354971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.354974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.355332 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.355591 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.355890 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.356142 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:43:49.357087 25674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:49.440747 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25538
I20260809 22:43:49.459079 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25685
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:49.476491 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.459999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.21.27.130:35693)
0809 22:43:49.460536 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.460543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.460568 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:43:49.460629 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.460636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.460856 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.460989 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.462191 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.463365 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.464780 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.465434 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.465538 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.470286 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.470298 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.470313 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.470377 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.473773 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.474541 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.474548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.474550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.474673 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.474991 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.474994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.474996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.475407 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.475643 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.475883 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.476081 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":283,"threads_started":1}
I20260809 22:43:49.477727 25821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:49.566836 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25685
I20260809 22:43:49.586107 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25831
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:49.603212 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:49.587307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.21.27.131:41695)
0809 22:43:49.587983 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:49.587990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:49.588019 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:43:49.588081 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:49.588088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:49.588304 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:49.588438 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:49.589706 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.590734 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.592303 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:49.592866 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:49.592967 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:49.597513 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:49.597528 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:49.597544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:49.597580 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:49.600497 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.601218 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.601225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.601227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.601348 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:49.601664 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:49.601667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:49.601669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:49.602028 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:49.602270 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:49.602534 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 22:43:49.602740 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":367,"threads_started":1}
I20260809 22:43:49.604610 25966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:49.688060 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25831
I20260809 22:43:49.705371 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 25437
I20260809 22:43:49.706677 25504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:49.815510 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 25437
2026-08-09T22:43:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4014 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25990](info): set up 2 sockets
Aug 09 22:43:49 dist-test-slave-chsk krb5kdc[25990](info): commencing operation
krb5kdc: starting...
W20260809 22:43:50.968684 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:43:51 dist-test-slave-chsk krb5kdc[25990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:43:51.214604 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42639
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:51.562970 26004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:51.563432 26004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:51.563633 26004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:51.571280 26004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:51.571395 26004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:51.571494 26004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:51.571596 26004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:51.571699 26004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:51.582122 26004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:51.583578 26004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:51.584982 26004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:51.597334 26013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:51.597447 26010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:51.597373 26011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:51.599038 26004 server_base.cc:1034] running on GCE node
I20260809 22:43:51.599956 26004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:51.601711 26004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:51.602969 26004 hybrid_clock.cc:648] HybridClock initialized: now 1786315431602924 us; error 45 us; skew 500 ppm
I20260809 22:43:51.603412 26004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:51 dist-test-slave-chsk krb5kdc[25990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:51.606984 26004 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:43:51.608400 26004 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:51.608551 26004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:51.608914 26004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:51.612314 26004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "e7704b4506ad4925984830acf441b5a6"
format_stamp: "Formatted at 2026-08-09 22:43:51 on dist-test-slave-chsk"
server_key: "eaa31c696e20c848c32d45ba48110c8a"
server_key_iv: "28be4e40c080b68178ecedc2d5b302dd"
server_key_version: "encryptionkey@0"
I20260809 22:43:51.613343 26004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "e7704b4506ad4925984830acf441b5a6"
format_stamp: "Formatted at 2026-08-09 22:43:51 on dist-test-slave-chsk"
server_key: "eaa31c696e20c848c32d45ba48110c8a"
server_key_iv: "28be4e40c080b68178ecedc2d5b302dd"
server_key_version: "encryptionkey@0"
I20260809 22:43:51.619268 26004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:43:51.623564 26019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:51.625234 26004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:43:51.625396 26004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "e7704b4506ad4925984830acf441b5a6"
format_stamp: "Formatted at 2026-08-09 22:43:51 on dist-test-slave-chsk"
server_key: "eaa31c696e20c848c32d45ba48110c8a"
server_key_iv: "28be4e40c080b68178ecedc2d5b302dd"
server_key_version: "encryptionkey@0"
I20260809 22:43:51.625602 26004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:51.649439 26004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:51.653959 26004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:51.654206 26004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:51.675024 26004 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42639
I20260809 22:43:51.675002 26075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42639 every 8 connection(s)
I20260809 22:43:51.676030 26004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:51.679630 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26004
I20260809 22:43:51.679867 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:51.680028 26076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:51.680581 21614 external_mini_cluster.cc:1428] Setting key c0893643440ae262e9076f90623b26a0
I20260809 22:43:51.694763 26076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: Bootstrap starting.
Aug 09 22:43:51 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:51.698589 26076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:51.699932 26076 log.cc:824] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:51.703650 26076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: No bootstrap required, opened a new log
I20260809 22:43:51.705623 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:51.682646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.21.27.190:42639)
0809 22:43:51.683620 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:51.683640 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:51.683698 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:43:51.685375 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:51.685411 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:51.685932 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:51.686405 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:51.687956 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:51.689846 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:51.691307 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:51.692462 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:51.692678 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:51.697097 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:51.697108 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:51.697126 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:51.697176 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:51.700861 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:51.701802 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:51.701812 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:51.701820 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:51.702060 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:51.702553 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:51.702560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:51.702563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:51.703363 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:51.703681 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:51.704133 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 22:43:51.704606 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":428,"threads_started":1}
I20260809 22:43:51.712531 26076 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } }
I20260809 22:43:51.713079 26076 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:51.713229 26076 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7704b4506ad4925984830acf441b5a6, State: Initialized, Role: FOLLOWER
I20260809 22:43:51.713881 26076 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } }
I20260809 22:43:51.714141 26076 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:51.714300 26076 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:51.714483 26076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:51.716840 26076 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } }
I20260809 22:43:51.717384 26076 leader_election.cc:302] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7704b4506ad4925984830acf441b5a6; no voters:
I20260809 22:43:51.718037 26076 leader_election.cc:288] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:51.718209 26081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:51.719069 26081 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [term 1 LEADER]: Becoming Leader. State: Replica: e7704b4506ad4925984830acf441b5a6, State: Running, Role: LEADER
I20260809 22:43:51.719661 26081 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } }
I20260809 22:43:51.720551 26076 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:51.722900 26082 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7704b4506ad4925984830acf441b5a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } } }
I20260809 22:43:51.723351 26082 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:51.723201 26083 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7704b4506ad4925984830acf441b5a6. Latest consensus state: current_term: 1 leader_uuid: "e7704b4506ad4925984830acf441b5a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7704b4506ad4925984830acf441b5a6" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42639 } } }
I20260809 22:43:51.723637 26083 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:51.726285 26089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:51.733096 26089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:51.745594 26089 catalog_manager.cc:1347] Generated new cluster ID: 80790185811645e8afa7d384caa23402
I20260809 22:43:51.745874 26089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:51.764144 26089 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:51.765583 26089 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:51.777634 26089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: Generated new TSK 0
I20260809 22:43:51.778527 26089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:43:51.835287 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:52.184417 26104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:52.184722 26104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:52.184888 26104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:52.192373 26104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:52.192492 26104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:52.192684 26104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:52.202721 26104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:52.204176 26104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:52.205509 26104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:52.218585 26109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:52.218649 26110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:52.218618 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:52.219470 26104 server_base.cc:1034] running on GCE node
I20260809 22:43:52.220120 26104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:52.221159 26104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:52.222405 26104 hybrid_clock.cc:648] HybridClock initialized: now 1786315432222362 us; error 45 us; skew 500 ppm
I20260809 22:43:52.222739 26104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:52 dist-test-slave-chsk krb5kdc[25990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:52.226073 26104 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:43:52.227411 26104 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:52.227524 26104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:52.227810 26104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:52.230971 26104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "c515f9718a104004a5a7f3a66c0f1b72"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "a85aafa4570668a7f977afbd1c8cc331"
server_key_iv: "61c50d06c3d712daea868e56eb7127a3"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.231817 26104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "c515f9718a104004a5a7f3a66c0f1b72"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "a85aafa4570668a7f977afbd1c8cc331"
server_key_iv: "61c50d06c3d712daea868e56eb7127a3"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.237056 26104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:43:52.240708 26119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.242033 26104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:43:52.242228 26104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "c515f9718a104004a5a7f3a66c0f1b72"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "a85aafa4570668a7f977afbd1c8cc331"
server_key_iv: "61c50d06c3d712daea868e56eb7127a3"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.242449 26104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:52.265350 26104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:52.269829 26104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:52.270126 26104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:52.271229 26104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:52.272619 26104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:52.272734 26104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.272868 26104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:52.272966 26104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.306649 26104 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:45409
I20260809 22:43:52.306744 26245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:45409 every 8 connection(s)
I20260809 22:43:52.307574 26104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:52.312129 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26104
I20260809 22:43:52.312386 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:52.313217 21614 external_mini_cluster.cc:1428] Setting key 8270858e7d2c428dd35d859736a6e91b
Aug 09 22:43:52 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:52.329813 26248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:52.310071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:39603 (local address 127.21.27.190:42639)
0809 22:43:52.310495 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:52.310500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:52.311530 (+ 1030us) server_negotiation.cc:407] Connection header received
0809 22:43:52.314222 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:52.314228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:52.314350 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:52.314467 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:52.316160 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.317394 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.319001 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.319758 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.319915 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:52.323728 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:52.323734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:52.323737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:52.323772 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:52.326464 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:52.327387 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:52.327391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:52.327394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:52.327516 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:52.327963 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:52.327975 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:52.327977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:52.328263 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:52.328425 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:52.329166 (+ 741us) server_negotiation.cc:299] Negotiation successful
0809 22:43:52.329432 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":211,"threads_started":1}
I20260809 22:43:52.330888 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:52.310617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42639 (local address 127.21.27.129:39603)
0809 22:43:52.311363 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:52.311411 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:52.313929 (+ 2518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:52.314688 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:52.314698 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:52.315403 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:52.315971 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:52.315989 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.317560 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.317564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:52.318824 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:52.318836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.319976 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.319981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:52.320177 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:52.320983 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:52.321006 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:52.323488 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:52.326633 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:52.326642 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:52.326659 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:52.327252 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:52.327628 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:52.327634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:52.327637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:52.327835 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:52.328397 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:52.328405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:52.328765 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 22:43:52.329041 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 22:43:52.329633 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":290,"threads_started":1}
I20260809 22:43:52.332715 26246 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42639
I20260809 22:43:52.333104 26246 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:52.333989 26246 heartbeater.cc:511] Master 127.21.27.190:42639 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:43:52.336532 26038 ts_manager.cc:194] Registered new tserver with Master: c515f9718a104004a5a7f3a66c0f1b72 (127.21.27.129:45409)
I20260809 22:43:52.338078 26038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:39603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:43:52.364579 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:52.686479 26252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:52.686744 26252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:52.686908 26252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:52.694177 26252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:52.694284 26252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:52.694484 26252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:52.704018 26252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:52.705252 26252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:52.706561 26252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:52.719312 26257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:52.719223 26260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:52.719271 26258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:52.720366 26252 server_base.cc:1034] running on GCE node
I20260809 22:43:52.720999 26252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:52.721902 26252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:52.723179 26252 hybrid_clock.cc:648] HybridClock initialized: now 1786315432723107 us; error 54 us; skew 500 ppm
I20260809 22:43:52.723498 26252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:52 dist-test-slave-chsk krb5kdc[25990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:52.730430 26252 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:43:52.731599 26252 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:52.731693 26252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:52.731969 26252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:52.735083 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "df9bc6b92cfc496a98e96c5767cb2422"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "c4bf47fbef949a6540a611ceab07d238"
server_key_iv: "9da0c5b29f6c21ddc4bedca0300bddf0"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.736001 26252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "df9bc6b92cfc496a98e96c5767cb2422"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "c4bf47fbef949a6540a611ceab07d238"
server_key_iv: "9da0c5b29f6c21ddc4bedca0300bddf0"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.741103 26252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:43:52.744560 26267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.745877 26252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:43:52.746053 26252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "df9bc6b92cfc496a98e96c5767cb2422"
format_stamp: "Formatted at 2026-08-09 22:43:52 on dist-test-slave-chsk"
server_key: "c4bf47fbef949a6540a611ceab07d238"
server_key_iv: "9da0c5b29f6c21ddc4bedca0300bddf0"
server_key_version: "encryptionkey@0"
I20260809 22:43:52.746246 26252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:52.787364 26252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:52.791258 26252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:52.791498 26252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:52.792466 26252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:52.793787 26252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:52.793877 26252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.793958 26252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:52.794021 26252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:52.828181 26252 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:38793
I20260809 22:43:52.828274 26390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:38793 every 8 connection(s)
I20260809 22:43:52.829125 26252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:52.837375 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26252
I20260809 22:43:52.837591 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:52.838055 21614 external_mini_cluster.cc:1428] Setting key ee956dd1c5beb04f6a8c3be4812df812
Aug 09 22:43:52 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315432, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:52.849720 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:52.831721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:55137 (local address 127.21.27.190:42639)
0809 22:43:52.832165 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:52.832170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:52.833169 (+ 999us) server_negotiation.cc:407] Connection header received
0809 22:43:52.835131 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:52.835136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:52.835253 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:52.835375 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:52.836877 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.837753 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.839566 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.840062 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.840204 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:52.843747 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:52.843752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:52.843755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:52.843788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:52.846525 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:52.847343 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:52.847349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:52.847352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:52.847487 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:52.847933 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:52.847938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:52.847941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:52.848224 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:52.848381 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:52.849095 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 22:43:52.849323 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":217,"threads_started":1}
I20260809 22:43:52.850764 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:52.832314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42639 (local address 127.21.27.130:55137)
0809 22:43:52.833011 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:52.833056 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:52.834861 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:52.835604 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:52.835611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:52.836145 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:52.836695 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:52.836712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.837915 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.837919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:52.839412 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:52.839425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:52.840302 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:52.840310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:52.840528 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:52.841235 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:52.841256 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:52.843502 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:52.846672 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:52.846681 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:52.846698 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:52.847240 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:52.847601 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:52.847606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:52.847610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:52.847809 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:52.848358 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:52.848366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:52.848713 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 22:43:52.848975 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 22:43:52.849542 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":286,"threads_started":1}
I20260809 22:43:52.852339 26391 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42639
I20260809 22:43:52.852649 26391 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:52.853314 26391 heartbeater.cc:511] Master 127.21.27.190:42639 requested a full tablet report, sending...
I20260809 22:43:52.854807 26038 ts_manager.cc:194] Registered new tserver with Master: df9bc6b92cfc496a98e96c5767cb2422 (127.21.27.130:38793)
I20260809 22:43:52.856093 26038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:55137
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:43:52.886438 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:43:53.210958 26399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:53.211282 26399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:53.211378 26399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:53.218274 26399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:53.218392 26399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:53.218503 26399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:53.228365 26399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:53.229708 26399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:53.231056 26399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:53.244297 26404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:53.244311 26405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:53.244454 26407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:53.245213 26399 server_base.cc:1034] running on GCE node
I20260809 22:43:53.245820 26399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:53.246855 26399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:53.248081 26399 hybrid_clock.cc:648] HybridClock initialized: now 1786315433248017 us; error 80 us; skew 500 ppm
I20260809 22:43:53.248415 26399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:43:53 dist-test-slave-chsk krb5kdc[25990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:43:53.251561 26399 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:43:53.252677 26399 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:53.252827 26399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:53.253131 26399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:53.256080 26399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "697ed0a999124ad4abab42038dfa1e24"
format_stamp: "Formatted at 2026-08-09 22:43:53 on dist-test-slave-chsk"
server_key: "b2fa39bb4e61a5c8daaad1110b265dce"
server_key_iv: "d23ec42d2bb860326d524ac4b228d684"
server_key_version: "encryptionkey@0"
I20260809 22:43:53.256904 26399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "697ed0a999124ad4abab42038dfa1e24"
format_stamp: "Formatted at 2026-08-09 22:43:53 on dist-test-slave-chsk"
server_key: "b2fa39bb4e61a5c8daaad1110b265dce"
server_key_iv: "d23ec42d2bb860326d524ac4b228d684"
server_key_version: "encryptionkey@0"
I20260809 22:43:53.262705 26399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:53.266114 26414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:53.267405 26399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:53.267577 26399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "697ed0a999124ad4abab42038dfa1e24"
format_stamp: "Formatted at 2026-08-09 22:43:53 on dist-test-slave-chsk"
server_key: "b2fa39bb4e61a5c8daaad1110b265dce"
server_key_iv: "d23ec42d2bb860326d524ac4b228d684"
server_key_version: "encryptionkey@0"
I20260809 22:43:53.267791 26399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:53.288674 26399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:53.292546 26399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:53.292831 26399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:53.293812 26399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:53.295128 26399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:53.295295 26399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:53.295432 26399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:53.295522 26399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:53.329375 26399 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:35539
I20260809 22:43:53.329438 26538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:35539 every 8 connection(s)
I20260809 22:43:53.330304 26399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:53.338076 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26399
I20260809 22:43:53.338325 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:53.338789 21614 external_mini_cluster.cc:1428] Setting key 98d01391644b8fe2f080fb3b210c77e4
I20260809 22:43:53.341198 26246 heartbeater.cc:503] Master 127.21.27.190:42639 was elected leader, sending a full tablet report...
Aug 09 22:43:53 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315433, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:43:53.351109 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.333015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:44847 (local address 127.21.27.190:42639)
0809 22:43:53.333229 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.333234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.334373 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 22:43:53.336389 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.336396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.336507 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.336622 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:53.338434 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.339302 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.341031 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.341653 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.341776 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.345315 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:53.345320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:53.345323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:53.345354 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:53.348019 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.348742 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.348749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.348752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.348889 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.349310 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.349318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.349321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.349670 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:53.349863 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.350536 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.350761 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:43:53.352092 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.333511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42639 (local address 127.21.27.131:44847)
0809 22:43:53.334209 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.334247 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.336111 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.336880 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.336887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.337400 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:53.338133 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.338161 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.339476 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.339480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.340883 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.340895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.341894 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.341902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.342133 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.342873 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:43:53.342904 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:53.345127 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:53.348172 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:53.348187 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:53.348199 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:53.348618 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:53.349001 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:43:53.349006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:43:53.349009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:43:53.349176 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:43:53.349807 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:53.349814 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:53.350139 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.350439 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.350975 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":281,"threads_started":1}
I20260809 22:43:53.353557 26539 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42639
I20260809 22:43:53.353899 26539 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:53.354542 26539 heartbeater.cc:511] Master 127.21.27.190:42639 requested a full tablet report, sending...
I20260809 22:43:53.355841 26037 ts_manager.cc:194] Registered new tserver with Master: 697ed0a999124ad4abab42038dfa1e24 (127.21.27.131:35539)
I20260809 22:43:53.356642 26037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:44847
I20260809 22:43:53.366241 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:43:53.382627 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.370719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.21.27.190:42639)
0809 22:43:53.370923 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.370932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.371463 (+ 531us) server_negotiation.cc:407] Connection header received
0809 22:43:53.371577 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.371580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.371663 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.371739 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:53.373248 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.373922 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.375137 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.375620 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.375714 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.377890 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:53.377894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:53.377897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:53.377926 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:53.380310 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.380929 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.380933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.380935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.381047 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.381350 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.381353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.381354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.381640 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:53.381816 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.382114 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.382267 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:43:53.386621 26037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:53.393425 26037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:53.419533 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.405174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45409 (local address 127.0.0.1:52412)
0809 22:43:53.405994 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.406040 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.406285 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.408386 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.408391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.408423 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:53.409038 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.409066 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.411914 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.411922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.415253 (+ 3331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.415275 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.418071 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.418082 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.418394 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.418461 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.418602 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.418861 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":32,"thread_start_us":1500,"threads_started":1}
I20260809 22:43:53.421458 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.405353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52412 (local address 127.21.27.129:45409)
0809 22:43:53.407767 (+ 2414us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.407773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.407820 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:43:53.407923 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.407931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.408171 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.409313 (+ 1142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:53.409587 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.411713 (+ 2126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.415586 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.417706 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.420561 (+ 2855us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.420810 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.420892 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.421061 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2267,"thread_start_us":2093,"threads_started":1}
I20260809 22:43:53.422272 26553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.405276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:38793 (local address 127.0.0.1:45668)
0809 22:43:53.406976 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.406998 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.407133 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.407975 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.407983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.408025 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:53.408622 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.408643 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.414123 (+ 5480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.414131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.419235 (+ 5104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.419255 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.421333 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.421339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.421502 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.421537 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.421655 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.421795 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"mutex_wait_us":1,"thread_start_us":728,"threads_started":1}
I20260809 22:43:53.422326 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.405793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45668 (local address 127.21.27.130:38793)
0809 22:43:53.407348 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.407356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.407384 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:43:53.407480 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.407487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.407742 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.410343 (+ 2601us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:53.410641 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.412655 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.419509 (+ 6854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.421098 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.421274 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.421474 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.421830 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.421993 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1410,"thread_start_us":1231,"threads_started":1}
I20260809 22:43:53.425148 26173 tablet_service.cc:1467] Processing CreateTablet for tablet bd59a48f5c784b03a60e0585d0660ded (DEFAULT_TABLE table=test-table [id=e065ed87bbd04493a5f049303735190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:53.425407 26318 tablet_service.cc:1467] Processing CreateTablet for tablet bd59a48f5c784b03a60e0585d0660ded (DEFAULT_TABLE table=test-table [id=e065ed87bbd04493a5f049303735190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:53.426585 26173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd59a48f5c784b03a60e0585d0660ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:53.426759 26318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd59a48f5c784b03a60e0585d0660ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:53.431299 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.407901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35539 (local address 127.0.0.1:43562)
0809 22:43:53.419870 (+ 11969us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.419899 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.420041 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.421750 (+ 1709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.421755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.421780 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:53.421958 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.421969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.426587 (+ 4618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.426592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.428532 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.428547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.430473 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.430480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.430674 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.430710 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.430813 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.430953 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11806,"thread_start_us":12122,"threads_started":1}
I20260809 22:43:53.431533 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.408286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43562 (local address 127.21.27.131:35539)
0809 22:43:53.421010 (+ 12724us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.421108 (+ 98us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.421144 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:43:53.421282 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.421293 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.421541 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.422165 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:53.422514 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.426427 (+ 3913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.428759 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.430283 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.430417 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.430643 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.430855 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.431027 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12558,"thread_start_us":12379,"threads_started":1}
I20260809 22:43:53.435451 26470 tablet_service.cc:1467] Processing CreateTablet for tablet bd59a48f5c784b03a60e0585d0660ded (DEFAULT_TABLE table=test-table [id=e065ed87bbd04493a5f049303735190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:53.436846 26470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd59a48f5c784b03a60e0585d0660ded. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:53.439644 26559 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Bootstrap starting.
I20260809 22:43:53.441627 26560 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Bootstrap starting.
I20260809 22:43:53.443383 26559 tablet_bootstrap.cc:654] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:53.444732 26559 log.cc:824] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:53.445264 26560 tablet_bootstrap.cc:654] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:53.446573 26560 log.cc:824] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:53.448225 26559 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: No bootstrap required, opened a new log
I20260809 22:43:53.448494 26559 ts_tablet_manager.cc:1397] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260809 22:43:53.449542 26560 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: No bootstrap required, opened a new log
I20260809 22:43:53.449890 26560 ts_tablet_manager.cc:1397] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260809 22:43:53.450345 26564 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Bootstrap starting.
I20260809 22:43:53.453064 26564 tablet_bootstrap.cc:654] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:53.453962 26564 log.cc:824] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:53.456775 26564 tablet_bootstrap.cc:492] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: No bootstrap required, opened a new log
I20260809 22:43:53.457055 26564 ts_tablet_manager.cc:1397] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:43:53.459743 26559 raft_consensus.cc:348] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.460199 26559 raft_consensus.cc:374] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:53.460325 26559 raft_consensus.cc:729] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c515f9718a104004a5a7f3a66c0f1b72, State: Initialized, Role: FOLLOWER
I20260809 22:43:53.460564 26560 raft_consensus.cc:348] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.460973 26560 raft_consensus.cc:374] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:53.461079 26560 raft_consensus.cc:729] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9bc6b92cfc496a98e96c5767cb2422, State: Initialized, Role: FOLLOWER
I20260809 22:43:53.460954 26559 consensus_queue.cc:260] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.461901 26560 consensus_queue.cc:260] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.463027 26559 ts_tablet_manager.cc:1428] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260809 22:43:53.464082 26391 heartbeater.cc:503] Master 127.21.27.190:42639 was elected leader, sending a full tablet report...
I20260809 22:43:53.465771 26560 ts_tablet_manager.cc:1428] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Time spent starting tablet: real 0.016s user 0.008s sys 0.006s
I20260809 22:43:53.465936 26564 raft_consensus.cc:348] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.466333 26564 raft_consensus.cc:374] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:53.466456 26564 raft_consensus.cc:729] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697ed0a999124ad4abab42038dfa1e24, State: Initialized, Role: FOLLOWER
I20260809 22:43:53.467015 26564 consensus_queue.cc:260] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.468304 26539 heartbeater.cc:503] Master 127.21.27.190:42639 was elected leader, sending a full tablet report...
I20260809 22:43:53.468811 26564 ts_tablet_manager.cc:1428] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260809 22:43:53.559928 26247 tablet.cc:2369] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:43:53.581308 26392 tablet.cc:2369] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:43:53.582033 26540 tablet.cc:2369] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:53.634303 26567 raft_consensus.cc:482] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:53.634644 26567 raft_consensus.cc:504] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.636584 26567 leader_election.cc:288] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c515f9718a104004a5a7f3a66c0f1b72 (127.21.27.129:45409), 697ed0a999124ad4abab42038dfa1e24 (127.21.27.131:35539)
I20260809 22:43:53.643921 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.636932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:60919 (local address 127.21.27.129:45409)
0809 22:43:53.637158 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.637166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.638062 (+ 896us) server_negotiation.cc:407] Connection header received
0809 22:43:53.638238 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.638244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.638324 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.638435 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:53.639062 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.639877 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.641798 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.642950 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.643180 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.643257 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.643544 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.643669 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:43:53.644042 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.637158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:40579 (local address 127.21.27.131:35539)
0809 22:43:53.637400 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.637408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.638245 (+ 837us) server_negotiation.cc:407] Connection header received
0809 22:43:53.638441 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.638446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.638521 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.638626 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:43:53.639416 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.640089 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.642038 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.643062 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.643269 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.643350 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.643658 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.643788 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:43:53.644094 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.636899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45409 (local address 127.21.27.130:60919)
0809 22:43:53.637948 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.637964 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.638090 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.638475 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.638479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.638502 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:53.638880 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.638893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.640047 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.640052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.641587 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.641600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.643206 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.643214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.643377 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.643403 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.643518 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.643653 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":3,"thread_start_us":664,"threads_started":1}
I20260809 22:43:53.644196 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.637053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35539 (local address 127.21.27.130:40579)
0809 22:43:53.638099 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:53.638116 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:53.638284 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:53.638808 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:53.638812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:53.638842 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:43:53.639273 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.639281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.640238 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.640243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.641857 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:53.641872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.643316 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.643322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:53.643490 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.643515 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:43:53.643619 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:43:53.643741 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":22,"thread_start_us":663,"threads_started":1}
I20260809 22:43:53.645130 26490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd59a48f5c784b03a60e0585d0660ded" candidate_uuid: "df9bc6b92cfc496a98e96c5767cb2422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697ed0a999124ad4abab42038dfa1e24" is_pre_election: true
I20260809 22:43:53.645292 26195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd59a48f5c784b03a60e0585d0660ded" candidate_uuid: "df9bc6b92cfc496a98e96c5767cb2422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c515f9718a104004a5a7f3a66c0f1b72" is_pre_election: true
I20260809 22:43:53.645712 26490 raft_consensus.cc:2454] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df9bc6b92cfc496a98e96c5767cb2422 in term 0.
I20260809 22:43:53.645836 26195 raft_consensus.cc:2454] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df9bc6b92cfc496a98e96c5767cb2422 in term 0.
I20260809 22:43:53.646415 26272 leader_election.cc:302] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 697ed0a999124ad4abab42038dfa1e24, df9bc6b92cfc496a98e96c5767cb2422; no voters:
I20260809 22:43:53.646811 26567 raft_consensus.cc:2781] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:53.647015 26567 raft_consensus.cc:482] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:53.647145 26567 raft_consensus.cc:3037] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:53.649525 26567 raft_consensus.cc:504] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.650274 26567 leader_election.cc:288] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [CANDIDATE]: Term 1 election: Requested vote from peers c515f9718a104004a5a7f3a66c0f1b72 (127.21.27.129:45409), 697ed0a999124ad4abab42038dfa1e24 (127.21.27.131:35539)
I20260809 22:43:53.650754 26195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd59a48f5c784b03a60e0585d0660ded" candidate_uuid: "df9bc6b92cfc496a98e96c5767cb2422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c515f9718a104004a5a7f3a66c0f1b72"
I20260809 22:43:53.650797 26490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd59a48f5c784b03a60e0585d0660ded" candidate_uuid: "df9bc6b92cfc496a98e96c5767cb2422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697ed0a999124ad4abab42038dfa1e24"
I20260809 22:43:53.650986 26490 raft_consensus.cc:3037] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:53.650974 26195 raft_consensus.cc:3037] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:53.653214 26490 raft_consensus.cc:2454] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df9bc6b92cfc496a98e96c5767cb2422 in term 1.
I20260809 22:43:53.653213 26195 raft_consensus.cc:2454] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df9bc6b92cfc496a98e96c5767cb2422 in term 1.
I20260809 22:43:53.653688 26269 leader_election.cc:302] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 697ed0a999124ad4abab42038dfa1e24, c515f9718a104004a5a7f3a66c0f1b72, df9bc6b92cfc496a98e96c5767cb2422; no voters:
I20260809 22:43:53.654067 26567 raft_consensus.cc:2781] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:53.654496 26567 raft_consensus.cc:686] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 1 LEADER]: Becoming Leader. State: Replica: df9bc6b92cfc496a98e96c5767cb2422, State: Running, Role: LEADER
I20260809 22:43:53.654996 26567 consensus_queue.cc:237] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } }
I20260809 22:43:53.659577 26038 catalog_manager.cc:5495] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 reported cstate change: term changed from 0 to 1, leader changed from <none> to df9bc6b92cfc496a98e96c5767cb2422 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "df9bc6b92cfc496a98e96c5767cb2422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df9bc6b92cfc496a98e96c5767cb2422" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 38793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 } health_report { overall_health: UNKNOWN } } }
I20260809 22:43:53.711769 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.706510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.21.27.130:38793)
0809 22:43:53.706744 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.706751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.706772 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:53.706892 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.706896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.706993 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.707152 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:43:53.707550 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.708254 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.709584 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.710007 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.710148 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.710621 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:43:53.711094 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:43:53.711314 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.711418 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.711502 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:43:53.719655 26490 raft_consensus.cc:1261] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 1 FOLLOWER]: Refusing update from remote peer df9bc6b92cfc496a98e96c5767cb2422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:53.719666 26195 raft_consensus.cc:1261] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 1 FOLLOWER]: Refusing update from remote peer df9bc6b92cfc496a98e96c5767cb2422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:53.720570 26571 consensus_queue.cc:1035] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697ed0a999124ad4abab42038dfa1e24" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:53.721017 26567 consensus_queue.cc:1035] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c515f9718a104004a5a7f3a66c0f1b72" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:53.732945 26574 mvcc.cc:205] Tried to move back new op lower bound from 7316748016506101760 to 7316748016255234048. Current Snapshot: MvccSnapshot[applied={T|T < 7316748016506101760}]
I20260809 22:43:53.739056 26575 mvcc.cc:205] Tried to move back new op lower bound from 7316748016506101760 to 7316748016255234048. Current Snapshot: MvccSnapshot[applied={T|T < 7316748016506101760}]
I20260809 22:43:53.751318 26038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:53.751608 26038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:53.755101 26038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7704b4506ad4925984830acf441b5a6: Sending DeleteTablet for 3 replicas of tablet bd59a48f5c784b03a60e0585d0660ded
I20260809 22:43:53.756319 26173 tablet_service.cc:1514] Processing DeleteTablet for tablet bd59a48f5c784b03a60e0585d0660ded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:53 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:52412
I20260809 22:43:53.756580 26318 tablet_service.cc:1514] Processing DeleteTablet for tablet bd59a48f5c784b03a60e0585d0660ded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:53 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:45668
I20260809 22:43:53.756580 26470 tablet_service.cc:1514] Processing DeleteTablet for tablet bd59a48f5c784b03a60e0585d0660ded with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:53 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43562
I20260809 22:43:53.758494 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26104
I20260809 22:43:53.782611 26587 tablet_replica.cc:330] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: stopping tablet replica
I20260809 22:43:53.787720 26587 raft_consensus.cc:2229] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:53.788621 26585 tablet_replica.cc:330] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: stopping tablet replica
I20260809 22:43:53.789131 26585 raft_consensus.cc:2229] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:53.789786 26585 raft_consensus.cc:2258] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:53.790633 26586 tablet_replica.cc:330] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: stopping tablet replica
I20260809 22:43:53.791025 26586 raft_consensus.cc:2229] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:53.791193 26587 raft_consensus.cc:2258] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:53.791553 26586 raft_consensus.cc:2258] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:53.793370 26587 ts_tablet_manager.cc:1905] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:53.795112 26586 ts_tablet_manager.cc:1905] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:43:53 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:43:53.797175 26585 ts_tablet_manager.cc:1905] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:53.801512 26587 ts_tablet_manager.cc:1918] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:53.801759 26587 log.cc:1196] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/bd59a48f5c784b03a60e0585d0660ded
I20260809 22:43:53.802469 26587 ts_tablet_manager.cc:1939] T bd59a48f5c784b03a60e0585d0660ded P df9bc6b92cfc496a98e96c5767cb2422: Deleting consensus metadata
I20260809 22:43:53.804447 26022 catalog_manager.cc:4841] TS df9bc6b92cfc496a98e96c5767cb2422 (127.21.27.130:38793): tablet bd59a48f5c784b03a60e0585d0660ded (table test-table [id=e065ed87bbd04493a5f049303735190d]) successfully deleted
I20260809 22:43:53.804714 26585 ts_tablet_manager.cc:1918] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:53.805974 26585 log.cc:1196] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/bd59a48f5c784b03a60e0585d0660ded
I20260809 22:43:53.806084 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.759559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.21.27.129:45409)
0809 22:43:53.759755 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.759765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.778724 (+ 18959us) server_negotiation.cc:407] Connection header received
0809 22:43:53.779351 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.779361 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.779459 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.779558 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:53.782921 (+ 3363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.784559 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.791626 (+ 7067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.792263 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.792396 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.798182 (+ 5786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:53.798195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:53.798212 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:53.798259 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:53.802436 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.803346 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.803355 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.803358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.803542 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.804007 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.804012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.804016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.804470 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:53.804802 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.805422 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.805664 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:43:53.806511 26585 ts_tablet_manager.cc:1939] T bd59a48f5c784b03a60e0585d0660ded P c515f9718a104004a5a7f3a66c0f1b72: Deleting consensus metadata
I20260809 22:43:53.807317 26241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:53.812374 26586 ts_tablet_manager.cc:1918] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:53.812553 26586 log.cc:1196] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/bd59a48f5c784b03a60e0585d0660ded
I20260809 22:43:53.813028 26586 ts_tablet_manager.cc:1939] T bd59a48f5c784b03a60e0585d0660ded P 697ed0a999124ad4abab42038dfa1e24: Deleting consensus metadata
I20260809 22:43:53.814918 26024 catalog_manager.cc:4841] TS 697ed0a999124ad4abab42038dfa1e24 (127.21.27.131:35539): tablet bd59a48f5c784b03a60e0585d0660ded (table test-table [id=e065ed87bbd04493a5f049303735190d]) successfully deleted
I20260809 22:43:53.899562 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26104
I20260809 22:43:53.899729 26020 catalog_manager.cc:4841] TS c515f9718a104004a5a7f3a66c0f1b72 (127.21.27.129:45409): tablet bd59a48f5c784b03a60e0585d0660ded (table test-table [id=e065ed87bbd04493a5f049303735190d]) successfully deleted
I20260809 22:43:53.918783 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26252
Aug 09 22:43:53 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:43:53.936817 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:53.919639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.21.27.130:38793)
0809 22:43:53.919847 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:53.919853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:53.919922 (+ 69us) server_negotiation.cc:407] Connection header received
0809 22:43:53.920122 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:53.920128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:53.920251 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:53.920367 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:53.921708 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.922959 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.924442 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:53.925167 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:53.925306 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:53.929772 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:53.929792 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:53.929814 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:53.929872 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:53.933967 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.934618 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.934629 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.934633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.934823 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:53.935198 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:53.935203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:53.935207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:53.935676 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:53.936002 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:53.936211 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 22:43:53.936463 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:43:53.937561 26386 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:54.052002 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26252
I20260809 22:43:54.073342 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26399
Aug 09 22:43:54 dist-test-slave-chsk krb5kdc[25990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:43:54.088616 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:54.074129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.21.27.131:35539)
0809 22:43:54.074336 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:54.074343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:54.074455 (+ 112us) server_negotiation.cc:407] Connection header received
0809 22:43:54.074667 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:54.074673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:54.074759 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:54.074866 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:54.076215 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:54.076924 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:54.078204 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:54.078630 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:54.078767 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:54.083103 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:54.083117 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:54.083132 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:43:54.083221 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:54.085803 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:54.086460 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:54.086467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:54.086469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:54.086601 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:43:54.086988 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:43:54.086995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:43:54.086998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:43:54.087424 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:54.087716 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:54.088034 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 22:43:54.088269 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:43:54.089265 26534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:54.177619 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26399
I20260809 22:43:54.196563 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26004
I20260809 22:43:54.197763 26071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:54.307621 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26004
2026-08-09T22:43:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4491 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T22:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:54Z Disabled control of system clock
I20260809 22:43:54.380751 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34105
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:34105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:43:54.710680 26598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:54.710964 26598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:54.711102 26598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:54.718349 26598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:54.718461 26598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:54.718499 26598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:54.718575 26598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:54.718643 26598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:43:54.728554 26598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:34105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:34105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:54.729945 26598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:54.731792 26598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:54.743673 26603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:54.743682 26607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:54.743649 26604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:54.744593 26598 server_base.cc:1034] running on GCE node
I20260809 22:43:54.745419 26598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:54.746779 26598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:54.748008 26598 hybrid_clock.cc:648] HybridClock initialized: now 1786315434747967 us; error 51 us; skew 500 ppm
I20260809 22:43:54.748370 26598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:54.750702 26598 webserver.cc:460] Webserver started at http://127.21.27.190:42827/ using document root <none> and password file <none>
I20260809 22:43:54.751534 26598 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:54.751670 26598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:54.752012 26598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:54.754911 26598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "846621a9033d4f4097eebd03da164852"
format_stamp: "Formatted at 2026-08-09 22:43:54 on dist-test-slave-chsk"
server_key: "1cff6f26b88fca37299ae0c1caee5393"
server_key_iv: "921672bb655c311f798b995861138f87"
server_key_version: "encryptionkey@0"
I20260809 22:43:54.755779 26598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "846621a9033d4f4097eebd03da164852"
format_stamp: "Formatted at 2026-08-09 22:43:54 on dist-test-slave-chsk"
server_key: "1cff6f26b88fca37299ae0c1caee5393"
server_key_iv: "921672bb655c311f798b995861138f87"
server_key_version: "encryptionkey@0"
I20260809 22:43:54.760820 26598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:43:54.764199 26614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:54.765465 26598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:43:54.765648 26598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "846621a9033d4f4097eebd03da164852"
format_stamp: "Formatted at 2026-08-09 22:43:54 on dist-test-slave-chsk"
server_key: "1cff6f26b88fca37299ae0c1caee5393"
server_key_iv: "921672bb655c311f798b995861138f87"
server_key_version: "encryptionkey@0"
I20260809 22:43:54.765889 26598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:54.771764 26598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:54.772519 26598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:54.772820 26598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:54.791399 26598 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:34105
I20260809 22:43:54.791440 26671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:34105 every 8 connection(s)
I20260809 22:43:54.792771 26598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:54.796556 26672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:54.801195 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26598
I20260809 22:43:54.801394 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:54.802269 21614 external_mini_cluster.cc:1428] Setting key 36d5450c92a5e01d03b0caebe0c479b9
I20260809 22:43:54.808825 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: Bootstrap starting.
I20260809 22:43:54.811416 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:54.804409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.21.27.190:34105)
0809 22:43:54.805725 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:54.805737 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:54.805780 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:43:54.807604 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:54.807627 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:54.808067 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:54.808482 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:54.809657 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:54.809662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:54.809674 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:54.809693 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:54.809744 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:54.809758 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:54.810008 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:54.810161 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 22:43:54.810419 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":584,"threads_started":1}
I20260809 22:43:54.812459 26672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:54.813581 26672 log.cc:824] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:54.816447 26672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: No bootstrap required, opened a new log
I20260809 22:43:54.823916 26672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } }
I20260809 22:43:54.824288 26672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:54.824414 26672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 846621a9033d4f4097eebd03da164852, State: Initialized, Role: FOLLOWER
I20260809 22:43:54.825032 26672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } }
I20260809 22:43:54.825264 26672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:54.825395 26672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:54.825575 26672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:54.828168 26672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } }
I20260809 22:43:54.828701 26672 leader_election.cc:302] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 846621a9033d4f4097eebd03da164852; no voters:
I20260809 22:43:54.829375 26672 leader_election.cc:288] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:54.829553 26677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:54.830286 26677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [term 1 LEADER]: Becoming Leader. State: Replica: 846621a9033d4f4097eebd03da164852, State: Running, Role: LEADER
I20260809 22:43:54.830967 26677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } }
I20260809 22:43:54.831718 26672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:54.834031 26679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 846621a9033d4f4097eebd03da164852. Latest consensus state: current_term: 1 leader_uuid: "846621a9033d4f4097eebd03da164852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } } }
I20260809 22:43:54.834476 26679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:54.834576 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "846621a9033d4f4097eebd03da164852" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "846621a9033d4f4097eebd03da164852" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 34105 } } }
I20260809 22:43:54.834924 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:54.836645 26686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:54.842063 26686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:54.850562 26686 catalog_manager.cc:1347] Generated new cluster ID: a10b11ec771145dab02639ca2439a68c
I20260809 22:43:54.850698 26686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:54.862618 26686 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:54.863128 26686 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:54.886601 26686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: Generated new TSK 0
I20260809 22:43:54.887686 26686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:43:54.898607 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--builtin_ntp_servers=127.21.27.148:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:43:55.238843 26696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:55.239298 26696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:55.239493 26696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:55.246754 26696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:55.246896 26696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:55.247062 26696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:43:55.256131 26696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:55.257396 26696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:55.259219 26696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:55.273638 26704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:55.273712 26701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:55.273729 26702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:55.274588 26696 server_base.cc:1034] running on GCE node
I20260809 22:43:55.275281 26696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:55.276328 26696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:55.277578 26696 hybrid_clock.cc:648] HybridClock initialized: now 1786315435277522 us; error 57 us; skew 500 ppm
I20260809 22:43:55.278003 26696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:55.280267 26696 webserver.cc:460] Webserver started at http://127.21.27.129:39303/ using document root <none> and password file <none>
I20260809 22:43:55.280969 26696 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:55.281107 26696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:55.281445 26696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:55.284376 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "6cae0e2c8470483789bc6f855aae47ca"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "71972d0d60814e39e9380ec04ef43330"
server_key_iv: "78baba0057ed0482a2e0d674926572ca"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.285171 26696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "6cae0e2c8470483789bc6f855aae47ca"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "71972d0d60814e39e9380ec04ef43330"
server_key_iv: "78baba0057ed0482a2e0d674926572ca"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.290205 26696 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:55.293620 26711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.294853 26696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:43:55.295018 26696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "6cae0e2c8470483789bc6f855aae47ca"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "71972d0d60814e39e9380ec04ef43330"
server_key_iv: "78baba0057ed0482a2e0d674926572ca"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.295303 26696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:55.300771 26696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:55.301537 26696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:55.301816 26696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:55.302704 26696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:55.304111 26696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:55.304234 26696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.304368 26696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:55.304457 26696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.338339 26696 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:43793
I20260809 22:43:55.338392 26832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:43793 every 8 connection(s)
I20260809 22:43:55.339928 26696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:43:55.341754 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26696
I20260809 22:43:55.341964 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:43:55.342828 21614 external_mini_cluster.cc:1428] Setting key 5bbd07274aab6413c31224ea64de191a
I20260809 22:43:55.345620 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--builtin_ntp_servers=127.21.27.148:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:43:55.348847 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:55.342524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48477 (local address 127.21.27.190:34105)
0809 22:43:55.343029 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:55.343037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:55.344628 (+ 1591us) server_negotiation.cc:407] Connection header received
0809 22:43:55.346442 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:55.346448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:55.346517 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:55.346645 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:55.348039 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:55.348044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:55.348049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:55.348068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:55.348119 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:55.348130 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:55.348267 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:55.348477 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 22:43:55.348554 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260809 22:43:55.349731 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:55.343124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34105 (local address 127.21.27.129:48477)
0809 22:43:55.344483 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:55.344520 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:55.346170 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:55.346877 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:55.346885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:55.346973 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:55.347742 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:55.347745 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:55.347782 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:55.347867 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:55.348234 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:55.348246 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:55.348254 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:43:55.348546 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 22:43:55.348792 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":357,"threads_started":1}
I20260809 22:43:55.351100 26833 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34105
I20260809 22:43:55.351431 26833 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:55.352013 26833 heartbeater.cc:511] Master 127.21.27.190:34105 requested a full tablet report, sending...
I20260809 22:43:55.354267 26633 ts_manager.cc:194] Registered new tserver with Master: 6cae0e2c8470483789bc6f855aae47ca (127.21.27.129:43793)
W20260809 22:43:55.676507 26838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:55.676788 26838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:55.676935 26838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:55.684131 26838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:55.684231 26838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:55.684442 26838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:43:55.693326 26838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:55.694587 26838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:55.696342 26838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:55.709443 26843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:55.709784 26844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:55.709511 26846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:55.710747 26838 server_base.cc:1034] running on GCE node
I20260809 22:43:55.711412 26838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:55.712329 26838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:55.713560 26838 hybrid_clock.cc:648] HybridClock initialized: now 1786315435713527 us; error 52 us; skew 500 ppm
I20260809 22:43:55.713884 26838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:55.716223 26838 webserver.cc:460] Webserver started at http://127.21.27.130:33229/ using document root <none> and password file <none>
I20260809 22:43:55.716900 26838 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:55.717031 26838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:55.717358 26838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:55.720191 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "0244b18ce7b9437a947054261aefa1d4"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "84ed9ead43e22b48c296552e7f702fa4"
server_key_iv: "066bc52c4201f2b5e447cf054cc5feab"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.720980 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "0244b18ce7b9437a947054261aefa1d4"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "84ed9ead43e22b48c296552e7f702fa4"
server_key_iv: "066bc52c4201f2b5e447cf054cc5feab"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.725956 26838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:43:55.731769 26852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.733247 26838 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:43:55.733448 26838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "0244b18ce7b9437a947054261aefa1d4"
format_stamp: "Formatted at 2026-08-09 22:43:55 on dist-test-slave-chsk"
server_key: "84ed9ead43e22b48c296552e7f702fa4"
server_key_iv: "066bc52c4201f2b5e447cf054cc5feab"
server_key_version: "encryptionkey@0"
I20260809 22:43:55.733698 26838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:55.741314 26838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:55.742175 26838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:55.742463 26838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:55.743458 26838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:55.744891 26838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:55.745004 26838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.745131 26838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:55.745231 26838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:55.778388 26838 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:35899
I20260809 22:43:55.778445 26975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:35899 every 8 connection(s)
I20260809 22:43:55.779834 26838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:43:55.780123 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26838
I20260809 22:43:55.780292 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:43:55.781016 21614 external_mini_cluster.cc:1428] Setting key aec7b48769c80162e8bc7f04555a058e
I20260809 22:43:55.784490 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--builtin_ntp_servers=127.21.27.148:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:43:55.788618 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:55.782683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:50507 (local address 127.21.27.190:34105)
0809 22:43:55.782872 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:55.782877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:55.784532 (+ 1655us) server_negotiation.cc:407] Connection header received
0809 22:43:55.786284 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:55.786289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:55.786358 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:55.786477 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:55.787847 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:55.787853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:55.787858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:55.787877 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:55.787920 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:55.787932 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:55.788058 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:55.788260 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 22:43:55.788335 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:43:55.789534 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:55.783427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34105 (local address 127.21.27.130:50507)
0809 22:43:55.784373 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:55.784412 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:55.786013 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:55.786721 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:55.786730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:55.786816 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:55.787554 (+ 738us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:55.787556 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:55.787594 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:55.787680 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:55.788031 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:55.788042 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:55.788050 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:43:55.788345 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 22:43:55.788593 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":312,"threads_started":1}
I20260809 22:43:55.790951 26976 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34105
I20260809 22:43:55.791296 26976 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:55.791982 26976 heartbeater.cc:511] Master 127.21.27.190:34105 requested a full tablet report, sending...
I20260809 22:43:55.793529 26633 ts_manager.cc:194] Registered new tserver with Master: 0244b18ce7b9437a947054261aefa1d4 (127.21.27.130:35899)
W20260809 22:43:56.117987 26980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:56.118305 26980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:56.118475 26980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:56.125799 26980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:56.125941 26980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:56.126127 26980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:43:56.135610 26980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:56.136952 26980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:56.138880 26980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:56.152177 26985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:56.152220 26986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:56.152254 26988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:56.153096 26980 server_base.cc:1034] running on GCE node
I20260809 22:43:56.153633 26980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:56.154491 26980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:56.155689 26980 hybrid_clock.cc:648] HybridClock initialized: now 1786315436155650 us; error 38 us; skew 500 ppm
I20260809 22:43:56.155964 26980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:56.158017 26980 webserver.cc:460] Webserver started at http://127.21.27.131:35243/ using document root <none> and password file <none>
I20260809 22:43:56.158627 26980 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:56.158720 26980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:56.158994 26980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:56.161756 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "073fff20b1794da08a47a8f96c8ae2bf"
format_stamp: "Formatted at 2026-08-09 22:43:56 on dist-test-slave-chsk"
server_key: "371c2d4b88ba1d6127656733eb5e47d5"
server_key_iv: "c48f22919a119a9d224a5ca41c393246"
server_key_version: "encryptionkey@0"
I20260809 22:43:56.162565 26980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "073fff20b1794da08a47a8f96c8ae2bf"
format_stamp: "Formatted at 2026-08-09 22:43:56 on dist-test-slave-chsk"
server_key: "371c2d4b88ba1d6127656733eb5e47d5"
server_key_iv: "c48f22919a119a9d224a5ca41c393246"
server_key_version: "encryptionkey@0"
I20260809 22:43:56.167477 26980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:43:56.170807 26994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:56.172080 26980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:43:56.172215 26980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "073fff20b1794da08a47a8f96c8ae2bf"
format_stamp: "Formatted at 2026-08-09 22:43:56 on dist-test-slave-chsk"
server_key: "371c2d4b88ba1d6127656733eb5e47d5"
server_key_iv: "c48f22919a119a9d224a5ca41c393246"
server_key_version: "encryptionkey@0"
I20260809 22:43:56.172379 26980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:56.177493 26980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:56.178158 26980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:56.178365 26980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:56.179203 26980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:43:56.180533 26980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:43:56.180617 26980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:56.180702 26980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:43:56.180756 26980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:56.212448 26980 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:33621
I20260809 22:43:56.212467 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:33621 every 8 connection(s)
I20260809 22:43:56.214119 26980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:43:56.220371 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 26980
I20260809 22:43:56.220546 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:43:56.221123 21614 external_mini_cluster.cc:1428] Setting key 1d360761a290374b0d4f4d19c1746dff
I20260809 22:43:56.222756 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.216895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:35619 (local address 127.21.27.190:34105)
0809 22:43:56.217078 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.217083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.218382 (+ 1299us) server_negotiation.cc:407] Connection header received
0809 22:43:56.220109 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.220113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.220166 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.220252 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.221792 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.221799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.221803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.221823 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.221867 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.221884 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.222026 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.222445 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.222509 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:43:56.223973 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.217357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:34105 (local address 127.21.27.131:35619)
0809 22:43:56.218224 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.218263 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.219818 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.220521 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.220536 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.220650 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.221404 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.221409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.221458 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.221569 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.222207 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.222215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.222220 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.222500 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.222780 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":285,"threads_started":1}
I20260809 22:43:56.225209 27115 heartbeater.cc:347] Connected to a master server at 127.21.27.190:34105
I20260809 22:43:56.225487 27115 heartbeater.cc:464] Registering TS with master...
I20260809 22:43:56.226006 27115 heartbeater.cc:511] Master 127.21.27.190:34105 requested a full tablet report, sending...
I20260809 22:43:56.227283 26633 ts_manager.cc:194] Registered new tserver with Master: 073fff20b1794da08a47a8f96c8ae2bf (127.21.27.131:33621)
I20260809 22:43:56.236598 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:43:56.243366 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.240966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.21.27.190:34105)
0809 22:43:56.241127 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.241131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.241799 (+ 668us) server_negotiation.cc:407] Connection header received
0809 22:43:56.241927 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.241930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.241979 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.242066 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.242834 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.242838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.242841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.242856 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.242885 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.242896 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.242990 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.243022 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.243063 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:43:56.247028 26633 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:43:56.253719 26633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:43:56.278767 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.265433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.21.27.130:35899)
0809 22:43:56.265978 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.265996 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.272282 (+ 6286us) server_negotiation.cc:407] Connection header received
0809 22:43:56.272367 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.272375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.272622 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.272794 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.276467 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.276481 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.276499 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.276524 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.276603 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.276636 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.276839 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.278137 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.278275 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":754,"threads_started":1}
I20260809 22:43:56.278873 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.265224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:35899 (local address 127.0.0.1:38922)
0809 22:43:56.271940 (+ 6716us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.271986 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.272131 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.274401 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.274413 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.274524 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.275827 (+ 1303us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.275833 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.275870 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.275977 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.277100 (+ 1123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.277115 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.277127 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.277321 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.278316 (+ 995us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":736,"threads_started":1}
I20260809 22:43:56.283454 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.266732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:33621 (local address 127.0.0.1:54064)
0809 22:43:56.279176 (+ 12444us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.279195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.279301 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.280318 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.280326 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.280383 (+ 57us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.281072 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.281075 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.281094 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.281144 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.283075 (+ 1931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.283085 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.283092 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.283207 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.283267 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12298,"thread_start_us":11087,"threads_started":1}
I20260809 22:43:56.284375 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.267122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.21.27.131:33621)
0809 22:43:56.277793 (+ 10671us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.277803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.279738 (+ 1935us) server_negotiation.cc:407] Connection header received
0809 22:43:56.279847 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.279857 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.280100 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.281564 (+ 1464us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.282781 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.282792 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.282809 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.282832 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.282899 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.282922 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.283776 (+ 854us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.283848 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.283943 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10486,"thread_start_us":10288,"threads_started":1}
I20260809 22:43:56.283924 26902 tablet_service.cc:1467] Processing CreateTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb (DEFAULT_TABLE table=test-table [id=4aab904f4fac45b4b7aaae2f160de5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:56.285490 26902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:56.286441 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.266021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48574)
0809 22:43:56.278059 (+ 12038us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.278078 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.278175 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.284054 (+ 5879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.284058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.284080 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.284746 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.284751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.284766 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.284809 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.285873 (+ 1064us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.285878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.285883 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.286000 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.286047 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11895,"thread_start_us":16982,"threads_started":1}
I20260809 22:43:56.287309 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.266602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.21.27.129:43793)
0809 22:43:56.283564 (+ 16962us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.283572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.283601 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:43:56.283667 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.283674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.283850 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.284445 (+ 595us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.285630 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.285637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.285649 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.285675 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.285724 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.285741 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.286803 (+ 1062us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.286876 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.286970 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16799,"thread_start_us":17220,"threads_started":1}
I20260809 22:43:56.287652 27045 tablet_service.cc:1467] Processing CreateTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb (DEFAULT_TABLE table=test-table [id=4aab904f4fac45b4b7aaae2f160de5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:56.288755 27045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:56.290485 26762 tablet_service.cc:1467] Processing CreateTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb (DEFAULT_TABLE table=test-table [id=4aab904f4fac45b4b7aaae2f160de5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:43:56.291751 26762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:56.301116 27137 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Bootstrap starting.
I20260809 22:43:56.301617 27138 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Bootstrap starting.
I20260809 22:43:56.304873 27138 tablet_bootstrap.cc:654] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:56.304868 27137 tablet_bootstrap.cc:654] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:56.305300 27139 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Bootstrap starting.
I20260809 22:43:56.306171 27137 log.cc:824] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:56.306677 27138 log.cc:824] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:56.308786 27139 tablet_bootstrap.cc:654] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:56.309513 27137 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: No bootstrap required, opened a new log
I20260809 22:43:56.309808 27137 ts_tablet_manager.cc:1397] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 22:43:56.309975 27139 log.cc:824] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:56.313021 27139 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: No bootstrap required, opened a new log
I20260809 22:43:56.313287 27139 ts_tablet_manager.cc:1397] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 22:43:56.314782 27138 tablet_bootstrap.cc:492] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: No bootstrap required, opened a new log
I20260809 22:43:56.315052 27138 ts_tablet_manager.cc:1397] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260809 22:43:56.320539 27137 raft_consensus.cc:348] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.321007 27137 raft_consensus.cc:374] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:56.321108 27137 raft_consensus.cc:729] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073fff20b1794da08a47a8f96c8ae2bf, State: Initialized, Role: FOLLOWER
I20260809 22:43:56.321750 27137 consensus_queue.cc:260] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.322993 27115 heartbeater.cc:503] Master 127.21.27.190:34105 was elected leader, sending a full tablet report...
I20260809 22:43:56.322978 27138 raft_consensus.cc:348] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.323426 27138 raft_consensus.cc:374] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:56.323535 27138 raft_consensus.cc:729] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0244b18ce7b9437a947054261aefa1d4, State: Initialized, Role: FOLLOWER
I20260809 22:43:56.323850 27139 raft_consensus.cc:348] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.324268 27139 raft_consensus.cc:374] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:56.324378 27139 raft_consensus.cc:729] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cae0e2c8470483789bc6f855aae47ca, State: Initialized, Role: FOLLOWER
I20260809 22:43:56.324262 27138 consensus_queue.cc:260] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.324707 27137 ts_tablet_manager.cc:1428] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260809 22:43:56.324975 27139 consensus_queue.cc:260] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.325767 26976 heartbeater.cc:503] Master 127.21.27.190:34105 was elected leader, sending a full tablet report...
I20260809 22:43:56.326248 26833 heartbeater.cc:503] Master 127.21.27.190:34105 was elected leader, sending a full tablet report...
I20260809 22:43:56.327714 27139 ts_tablet_manager.cc:1428] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 22:43:56.328330 27138 ts_tablet_manager.cc:1428] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260809 22:43:56.342274 26834 tablet.cc:2369] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:56.411425 27143 raft_consensus.cc:482] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:43:56.411720 27143 raft_consensus.cc:504] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.413403 27143 leader_election.cc:288] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0244b18ce7b9437a947054261aefa1d4 (127.21.27.130:35899), 6cae0e2c8470483789bc6f855aae47ca (127.21.27.129:43793)
I20260809 22:43:56.416186 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.413680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.21.27.131:40587)
0809 22:43:56.413866 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.413886 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.414003 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.414525 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.414530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.414556 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.415353 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.415355 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.415375 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.415412 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.415735 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.415739 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.415744 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.415854 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.415910 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:43:56.416185 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.414019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:40587 (local address 127.21.27.129:43793)
0809 22:43:56.414203 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.414209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.414230 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:56.414295 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.414301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.414370 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.414491 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.415572 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.415578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.415581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.415595 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.415623 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.415635 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.415754 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.415856 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.415911 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:43:56.416795 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.414126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:52755 (local address 127.21.27.130:35899)
0809 22:43:56.414294 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.414300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.414690 (+ 390us) server_negotiation.cc:407] Connection header received
0809 22:43:56.414783 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.414791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.414860 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.414949 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.415972 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.415977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.415982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.416000 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.416039 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.416050 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.416288 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.416462 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.416549 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:43:56.416901 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.413836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:35899 (local address 127.21.27.131:52755)
0809 22:43:56.414580 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.414594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:56.414692 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:56.415007 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:56.415012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:56.415036 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:43:56.415771 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:43:56.415774 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:43:56.415787 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:43:56.415822 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:43:56.416180 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:43:56.416186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:43:56.416192 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:43:56.416453 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 22:43:56.416504 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":437,"threads_started":1}
I20260809 22:43:56.417409 26784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fdaf2f5a334bfeb4529a5265d4c4cb" candidate_uuid: "073fff20b1794da08a47a8f96c8ae2bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cae0e2c8470483789bc6f855aae47ca" is_pre_election: true
I20260809 22:43:56.417980 26784 raft_consensus.cc:2454] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073fff20b1794da08a47a8f96c8ae2bf in term 0.
I20260809 22:43:56.417892 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fdaf2f5a334bfeb4529a5265d4c4cb" candidate_uuid: "073fff20b1794da08a47a8f96c8ae2bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0244b18ce7b9437a947054261aefa1d4" is_pre_election: true
I20260809 22:43:56.418344 26924 raft_consensus.cc:2454] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073fff20b1794da08a47a8f96c8ae2bf in term 0.
I20260809 22:43:56.418673 26995 leader_election.cc:302] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073fff20b1794da08a47a8f96c8ae2bf, 6cae0e2c8470483789bc6f855aae47ca; no voters:
I20260809 22:43:56.419083 27143 raft_consensus.cc:2781] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:43:56.419251 27143 raft_consensus.cc:482] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:43:56.419359 27143 raft_consensus.cc:3037] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:56.421514 27143 raft_consensus.cc:504] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.422101 27143 leader_election.cc:288] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [CANDIDATE]: Term 1 election: Requested vote from peers 0244b18ce7b9437a947054261aefa1d4 (127.21.27.130:35899), 6cae0e2c8470483789bc6f855aae47ca (127.21.27.129:43793)
I20260809 22:43:56.422441 26924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fdaf2f5a334bfeb4529a5265d4c4cb" candidate_uuid: "073fff20b1794da08a47a8f96c8ae2bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0244b18ce7b9437a947054261aefa1d4"
I20260809 22:43:56.422477 26784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88fdaf2f5a334bfeb4529a5265d4c4cb" candidate_uuid: "073fff20b1794da08a47a8f96c8ae2bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cae0e2c8470483789bc6f855aae47ca"
I20260809 22:43:56.422646 26924 raft_consensus.cc:3037] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:56.422662 26784 raft_consensus.cc:3037] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:56.424925 26924 raft_consensus.cc:2454] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073fff20b1794da08a47a8f96c8ae2bf in term 1.
I20260809 22:43:56.424937 26784 raft_consensus.cc:2454] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073fff20b1794da08a47a8f96c8ae2bf in term 1.
I20260809 22:43:56.425349 26998 leader_election.cc:302] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0244b18ce7b9437a947054261aefa1d4, 073fff20b1794da08a47a8f96c8ae2bf; no voters:
I20260809 22:43:56.425665 27143 raft_consensus.cc:2781] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:56.426082 27143 raft_consensus.cc:686] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 1 LEADER]: Becoming Leader. State: Replica: 073fff20b1794da08a47a8f96c8ae2bf, State: Running, Role: LEADER
I20260809 22:43:56.426609 27143 consensus_queue.cc:237] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } }
I20260809 22:43:56.431403 26632 catalog_manager.cc:5495] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 073fff20b1794da08a47a8f96c8ae2bf (127.21.27.131). New cstate: current_term: 1 leader_uuid: "073fff20b1794da08a47a8f96c8ae2bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073fff20b1794da08a47a8f96c8ae2bf" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33621 } health_report { overall_health: HEALTHY } } }
W20260809 22:43:56.465924 27116 tablet.cc:2369] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:43:56.477634 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.475447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54068 (local address 127.21.27.131:33621)
0809 22:43:56.475665 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.475672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.475851 (+ 179us) server_negotiation.cc:407] Connection header received
0809 22:43:56.476009 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.476012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.476092 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.476176 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.477063 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.477067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.477070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.477085 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.477116 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.477127 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.477235 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.477294 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.477340 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:43:56.485486 26784 raft_consensus.cc:1261] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 1 FOLLOWER]: Refusing update from remote peer 073fff20b1794da08a47a8f96c8ae2bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:56.485533 26924 raft_consensus.cc:1261] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 1 FOLLOWER]: Refusing update from remote peer 073fff20b1794da08a47a8f96c8ae2bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:43:56.486465 27148 consensus_queue.cc:1035] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0244b18ce7b9437a947054261aefa1d4" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:56.486814 27143 consensus_queue.cc:1035] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cae0e2c8470483789bc6f855aae47ca" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:43:56.514958 26633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:56.515273 26633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:43:56.518792 26633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 846621a9033d4f4097eebd03da164852: Sending DeleteTablet for 3 replicas of tablet 88fdaf2f5a334bfeb4529a5265d4c4cb
I20260809 22:43:56.520072 26762 tablet_service.cc:1514] Processing DeleteTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:56 UTC) from {username='slave'} at 127.0.0.1:48574
I20260809 22:43:56.520339 26902 tablet_service.cc:1514] Processing DeleteTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:56 UTC) from {username='slave'} at 127.0.0.1:38922
I20260809 22:43:56.520807 27045 tablet_service.cc:1514] Processing DeleteTablet for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:43:56 UTC) from {username='slave'} at 127.0.0.1:54064
I20260809 22:43:56.521212 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26696
I20260809 22:43:56.533195 27162 tablet_replica.cc:330] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca: stopping tablet replica
I20260809 22:43:56.533681 27162 raft_consensus.cc:2229] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:56.534257 27162 raft_consensus.cc:2258] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 6cae0e2c8470483789bc6f855aae47ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:56.535301 27163 tablet_replica.cc:330] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: stopping tablet replica
I20260809 22:43:56.535715 27163 raft_consensus.cc:2229] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:43:56.536255 27163 raft_consensus.cc:2258] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:56.536317 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.522980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.21.27.129:43793)
0809 22:43:56.523249 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.523259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.533415 (+ 10156us) server_negotiation.cc:407] Connection header received
0809 22:43:56.533914 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.533920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.534022 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.534104 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.535477 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.535483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.535487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.535508 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.535547 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.535563 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.535664 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.535935 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.536026 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:43:56.537431 26828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:56.537802 27164 tablet_replica.cc:330] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: stopping tablet replica
I20260809 22:43:56.538290 27164 raft_consensus.cc:2229] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:43:56.539208 27164 raft_consensus.cc:2258] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:43:56.541288 27163 ts_tablet_manager.cc:1905] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:56.541745 27164 ts_tablet_manager.cc:1905] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:43:56.549193 27164 ts_tablet_manager.cc:1918] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:56.549288 27163 ts_tablet_manager.cc:1918] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:43:56.549477 27163 log.cc:1196] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/88fdaf2f5a334bfeb4529a5265d4c4cb
I20260809 22:43:56.549501 27164 log.cc:1196] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/88fdaf2f5a334bfeb4529a5265d4c4cb
I20260809 22:43:56.550015 27163 ts_tablet_manager.cc:1939] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 0244b18ce7b9437a947054261aefa1d4: Deleting consensus metadata
I20260809 22:43:56.550276 27164 ts_tablet_manager.cc:1939] T 88fdaf2f5a334bfeb4529a5265d4c4cb P 073fff20b1794da08a47a8f96c8ae2bf: Deleting consensus metadata
I20260809 22:43:56.552124 26620 catalog_manager.cc:4841] TS 073fff20b1794da08a47a8f96c8ae2bf (127.21.27.131:33621): tablet 88fdaf2f5a334bfeb4529a5265d4c4cb (table test-table [id=4aab904f4fac45b4b7aaae2f160de5d3]) successfully deleted
I20260809 22:43:56.552664 26620 catalog_manager.cc:4841] TS 0244b18ce7b9437a947054261aefa1d4 (127.21.27.130:35899): tablet 88fdaf2f5a334bfeb4529a5265d4c4cb (table test-table [id=4aab904f4fac45b4b7aaae2f160de5d3]) successfully deleted
I20260809 22:43:56.631278 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26696
I20260809 22:43:56.652628 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26838
W20260809 22:43:56.652743 26615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.27.129:43793 (error 108)
I20260809 22:43:56.654801 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.654250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48596)
0809 22:43:56.654496 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.654627 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260809 22:43:56.655755 26615 catalog_manager.cc:4584] TS 6cae0e2c8470483789bc6f855aae47ca (127.21.27.129:43793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88fdaf2f5a334bfeb4529a5265d4c4cb: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
I20260809 22:43:56.657850 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.654408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.21.27.130:35899)
0809 22:43:56.655081 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.655088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.655109 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:43:56.655255 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.655261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.655409 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.655518 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.656860 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.656867 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.656872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.656892 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.656932 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.656950 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.657079 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.657470 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.657550 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538}
I20260809 22:43:56.658658 26971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:56.709578 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.709057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48608)
0809 22:43:56.709297 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.709417 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 22:43:56.710678 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.710315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48614)
0809 22:43:56.710468 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.710552 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:43:56.771319 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.770766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48618)
0809 22:43:56.770981 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.771086 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:43:56.772505 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.772083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48632)
0809 22:43:56.772261 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.772360 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 22:43:56.797307 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26838
I20260809 22:43:56.816515 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26980
I20260809 22:43:56.819742 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.817324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54078 (local address 127.21.27.131:33621)
0809 22:43:56.817515 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:56.817522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:56.817650 (+ 128us) server_negotiation.cc:407] Connection header received
0809 22:43:56.817879 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:56.817884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:56.817937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:56.818045 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:56.818983 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:56.818989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:56.818994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:56.819019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:56.819050 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:56.819065 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:56.819247 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:56.819338 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 22:43:56.819402 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:43:56.820551 27110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:56.853070 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.852486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48648)
0809 22:43:56.852746 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.852872 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 22:43:56.854188 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:56.853745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48658)
0809 22:43:56.853969 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:56.854051 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 22:43:56.938481 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26980
I20260809 22:43:56.959064 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 26598
I20260809 22:43:56.960399 26667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:57.066259 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:57.065654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:43793 (local address 127.0.0.1:48664)
0809 22:43:57.065923 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:57.066052 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:43793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:43:57.066352 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 26598
2026-08-09T22:43:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2754 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 22:43:57.107501 21614 mini_oidc.cc:152] Starting JWKS server
I20260809 22:43:57.110419 21614 webserver.cc:460] Webserver started at https://127.0.0.1:42247/ using document root <none> and password file <none>
I20260809 22:43:57.110786 21614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:43:57.111487 21614 webserver.cc:460] Webserver started at http://127.0.0.1:35209/ using document root <none> and password file <none>
2026-08-09T22:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:57Z Disabled control of system clock
I20260809 22:43:57.142050 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41973
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:41973
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/testchainca.cert with env {}
W20260809 22:43:57.467156 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:57.467512 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:57.467685 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:57.474742 27179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:57.474908 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:57.475018 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:57.475127 27179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:57.475230 27179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:43:57.475322 27179 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:43:57.475400 27179 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42247/jwks.json
I20260809 22:43:57.485409 27179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:41973
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:57.486701 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:57.488029 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:57.500293 27184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:57.500344 27187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:57.500372 27185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:57.501286 27179 server_base.cc:1034] running on GCE node
I20260809 22:43:57.502143 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:57.503578 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:57.504805 27179 hybrid_clock.cc:648] HybridClock initialized: now 1786315437504769 us; error 33 us; skew 500 ppm
I20260809 22:43:57.505146 27179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:57.507440 27179 webserver.cc:460] Webserver started at http://127.21.27.190:34939/ using document root <none> and password file <none>
I20260809 22:43:57.508411 27179 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:57.508545 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:57.508853 27179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:57.511814 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "a645abdd0eeb4d2caf6acc7167d61f4a"
format_stamp: "Formatted at 2026-08-09 22:43:57 on dist-test-slave-chsk"
server_key: "8e20cd8411c8ad812137c51993d9386d"
server_key_iv: "a08679009aff6cbfae574d66a8f9e204"
server_key_version: "encryptionkey@0"
I20260809 22:43:57.512653 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "a645abdd0eeb4d2caf6acc7167d61f4a"
format_stamp: "Formatted at 2026-08-09 22:43:57 on dist-test-slave-chsk"
server_key: "8e20cd8411c8ad812137c51993d9386d"
server_key_iv: "a08679009aff6cbfae574d66a8f9e204"
server_key_version: "encryptionkey@0"
I20260809 22:43:57.517741 27179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:43:57.521062 27193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:57.522408 27179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:43:57.522604 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "a645abdd0eeb4d2caf6acc7167d61f4a"
format_stamp: "Formatted at 2026-08-09 22:43:57 on dist-test-slave-chsk"
server_key: "8e20cd8411c8ad812137c51993d9386d"
server_key_iv: "a08679009aff6cbfae574d66a8f9e204"
server_key_version: "encryptionkey@0"
I20260809 22:43:57.522840 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:57.548568 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:57.549482 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:57.549822 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:57.567823 27179 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:41973
I20260809 22:43:57.567848 27250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:41973 every 8 connection(s)
I20260809 22:43:57.569275 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:57.570818 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27179
I20260809 22:43:57.571045 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:57.571951 21614 external_mini_cluster.cc:1428] Setting key a40ae7ae3be287ab0b1def33b9f31247
I20260809 22:43:57.574054 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:57.587381 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:57.573857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.21.27.190:41973)
0809 22:43:57.574945 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:57.574959 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:57.575011 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:43:57.577180 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:57.577207 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:57.577816 (+ 609us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:57.578232 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:57.578763 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.580810 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.582180 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.583208 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.583389 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:57.584561 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:57.584572 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:57.584590 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:57.585337 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:57.585434 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:57.585468 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:57.585584 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:57.585773 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 22:43:57.586103 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":401,"threads_started":1}
I20260809 22:43:57.588645 27251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a: Bootstrap starting.
I20260809 22:43:57.591200 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:43:57.592046 27251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:57.593253 27251 log.cc:824] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:57.596095 27251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a: No bootstrap required, opened a new log
I20260809 22:43:57.603047 27251 raft_consensus.cc:348] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } }
I20260809 22:43:57.603510 27251 raft_consensus.cc:374] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:57.603590 27251 raft_consensus.cc:729] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a645abdd0eeb4d2caf6acc7167d61f4a, State: Initialized, Role: FOLLOWER
I20260809 22:43:57.604269 27251 consensus_queue.cc:260] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } }
I20260809 22:43:57.604547 27251 raft_consensus.cc:388] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:57.604734 27251 raft_consensus.cc:482] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:57.604923 27251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:57.607120 27251 raft_consensus.cc:504] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } }
I20260809 22:43:57.607755 27251 leader_election.cc:302] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a645abdd0eeb4d2caf6acc7167d61f4a; no voters:
I20260809 22:43:57.608374 27251 leader_election.cc:288] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:57.608618 27257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:57.609787 27257 raft_consensus.cc:686] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [term 1 LEADER]: Becoming Leader. State: Replica: a645abdd0eeb4d2caf6acc7167d61f4a, State: Running, Role: LEADER
I20260809 22:43:57.610365 27257 consensus_queue.cc:237] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } }
I20260809 22:43:57.610854 27251 sys_catalog.cc:564] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:57.613395 27258 sys_catalog.cc:455] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } } }
I20260809 22:43:57.613379 27259 sys_catalog.cc:455] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a645abdd0eeb4d2caf6acc7167d61f4a. Latest consensus state: current_term: 1 leader_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645abdd0eeb4d2caf6acc7167d61f4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41973 } } }
I20260809 22:43:57.613958 27258 sys_catalog.cc:458] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:57.614022 27259 sys_catalog.cc:458] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:57.615108 27263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:57.621958 27263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:57.632031 27263 catalog_manager.cc:1347] Generated new cluster ID: 721e00c0531641e88a1b0e98ba0c76b6
I20260809 22:43:57.632228 27263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:57.661351 27263 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:57.663376 27263 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:57.681555 27263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a645abdd0eeb4d2caf6acc7167d61f4a: Generated new TSK 0
I20260809 22:43:57.682467 27263 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:43:57.729132 27283 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 22:43:57.729573 27283 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:43:57.730384 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:57.707353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35574 (local address 127.21.27.190:41973)
0809 22:43:57.707563 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:57.707572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:57.708066 (+ 494us) server_negotiation.cc:407] Connection header received
0809 22:43:57.708442 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:57.708450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:57.708543 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:57.708715 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:43:57.709243 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.710390 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.712180 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.712934 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.713068 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:57.713588 (+ 520us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:43:57.729671 (+ 16083us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:43:57.730070 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 22:43:57.730505 27255 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35574: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 22:43:57.733347 27281 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:43:57.707245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41973 (local address 127.0.0.1:35574)
0809 22:43:57.707963 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:57.707980 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:57.708129 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:57.708749 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:57.708752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:57.708762 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:43:57.709085 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:57.709093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.710564 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.710573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:57.712012 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:57.712024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:57.713110 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:57.713115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:57.713236 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:57.713294 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:43:57.730965 (+ 17671us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:43:57.731355 (+ 390us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.27.190:41973: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":390,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.27.190:41973: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 22:43:57.736339 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27179
I20260809 22:43:57.737623 27245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:57.833881 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27179
2026-08-09T22:43:57Z chronyd exiting
I20260809 22:43:57.886828 21614 test_util.cc:175] -----------------------------------------------
I20260809 22:43:57.887025 21614 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786315394803050-21614-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (784 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 22:43:57.891027 21614 mini_oidc.cc:152] Starting JWKS server
I20260809 22:43:57.892976 21614 webserver.cc:460] Webserver started at https://127.0.0.1:43619/ using document root <none> and password file <none>
I20260809 22:43:57.893270 21614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:43:57.893949 21614 webserver.cc:460] Webserver started at http://127.0.0.1:35359/ using document root <none> and password file <none>
2026-08-09T22:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:57Z Disabled control of system clock
I20260809 22:43:57.926254 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37871
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:43643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:37871
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43619/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/testchainca.cert with env {}
W20260809 22:43:58.248580 27291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:58.248906 27291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:58.249053 27291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:58.256186 27291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:58.256328 27291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:58.256407 27291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:58.256521 27291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:58.256594 27291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:43:58.256681 27291 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:43:58.256767 27291 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43619/jwks.json
I20260809 22:43:58.266306 27291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:37871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43619/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:58.267658 27291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:58.268909 27291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:58.281101 27297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:58.281947 27296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:58.282292 27299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:58.283437 27291 server_base.cc:1034] running on GCE node
I20260809 22:43:58.284287 27291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:58.285668 27291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:58.286904 27291 hybrid_clock.cc:648] HybridClock initialized: now 1786315438286864 us; error 60 us; skew 500 ppm
I20260809 22:43:58.287322 27291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:58.293985 27291 webserver.cc:460] Webserver started at http://127.21.27.190:42613/ using document root <none> and password file <none>
I20260809 22:43:58.294752 27291 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:58.294849 27291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:58.295126 27291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:58.297998 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "a1d70053f8c445adbe868b54fca4f616"
format_stamp: "Formatted at 2026-08-09 22:43:58 on dist-test-slave-chsk"
server_key: "38c387c68bc63b3c0c3126827b4e7401"
server_key_iv: "2c8ac68c3636ad4a8228231d2ab77b1b"
server_key_version: "encryptionkey@0"
I20260809 22:43:58.298746 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "a1d70053f8c445adbe868b54fca4f616"
format_stamp: "Formatted at 2026-08-09 22:43:58 on dist-test-slave-chsk"
server_key: "38c387c68bc63b3c0c3126827b4e7401"
server_key_iv: "2c8ac68c3636ad4a8228231d2ab77b1b"
server_key_version: "encryptionkey@0"
I20260809 22:43:58.303751 27291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:43:58.307068 27305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:58.308699 27291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:43:58.308838 27291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "a1d70053f8c445adbe868b54fca4f616"
format_stamp: "Formatted at 2026-08-09 22:43:58 on dist-test-slave-chsk"
server_key: "38c387c68bc63b3c0c3126827b4e7401"
server_key_iv: "2c8ac68c3636ad4a8228231d2ab77b1b"
server_key_version: "encryptionkey@0"
I20260809 22:43:58.309031 27291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:58.340340 27291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:58.341156 27291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:58.341421 27291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:58.359282 27291 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:37871
I20260809 22:43:58.359367 27362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:37871 every 8 connection(s)
I20260809 22:43:58.360741 27291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:58.364358 27363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:58.365864 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27291
I20260809 22:43:58.366075 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:58.366997 21614 external_mini_cluster.cc:1428] Setting key 12e9adeca1ec1116261b0ca851645e2b
I20260809 22:43:58.377668 27363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616: Bootstrap starting.
I20260809 22:43:58.380261 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:58.369051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.21.27.190:37871)
0809 22:43:58.370101 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:58.370122 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:58.370163 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:43:58.371772 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:58.371792 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:58.372218 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:58.372576 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:58.373084 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.374351 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.375698 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.376344 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.376535 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:58.377792 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:58.377805 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:58.377823 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:58.378372 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:58.378447 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:58.378467 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:58.378607 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:58.378824 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 22:43:58.379244 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":417,"threads_started":1}
I20260809 22:43:58.381093 27363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:58.382491 27363 log.cc:824] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:58.384331 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:43:58.385903 27363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616: No bootstrap required, opened a new log
I20260809 22:43:58.393764 27363 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } }
I20260809 22:43:58.394085 27363 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:58.394166 27363 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d70053f8c445adbe868b54fca4f616, State: Initialized, Role: FOLLOWER
I20260809 22:43:58.394694 27363 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } }
I20260809 22:43:58.394917 27363 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:58.395063 27363 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:58.395282 27363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:58.397434 27363 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } }
I20260809 22:43:58.397933 27363 leader_election.cc:302] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1d70053f8c445adbe868b54fca4f616; no voters:
I20260809 22:43:58.398536 27363 leader_election.cc:288] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:58.398667 27369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:58.399434 27369 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [term 1 LEADER]: Becoming Leader. State: Replica: a1d70053f8c445adbe868b54fca4f616, State: Running, Role: LEADER
I20260809 22:43:58.399999 27369 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } }
I20260809 22:43:58.400679 27363 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:58.402820 27371 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1d70053f8c445adbe868b54fca4f616. Latest consensus state: current_term: 1 leader_uuid: "a1d70053f8c445adbe868b54fca4f616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } } }
I20260809 22:43:58.403311 27371 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:58.404913 27377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:58.404816 27370 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1d70053f8c445adbe868b54fca4f616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d70053f8c445adbe868b54fca4f616" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37871 } } }
I20260809 22:43:58.405476 27370 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:58.410629 27377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:58.418942 27377 catalog_manager.cc:1347] Generated new cluster ID: 7b6d0ef770e249d29a651cac1e610d92
I20260809 22:43:58.419055 27377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:58.456281 27377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:58.457599 27377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:58.465046 27377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1d70053f8c445adbe868b54fca4f616: Generated new TSK 0
I20260809 22:43:58.465890 27377 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:43:58.508255 27396 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 22:43:58.508394 27396 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:43:58.509701 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:58.489327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.21.27.190:37871)
0809 22:43:58.489530 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:58.489537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:58.489981 (+ 444us) server_negotiation.cc:407] Connection header received
0809 22:43:58.490189 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:58.490195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:58.490288 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:58.490464 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:43:58.490928 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.492112 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.493450 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.494153 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.494280 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:58.494814 (+ 534us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:43:58.508978 (+ 14164us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:43:58.509376 (+ 398us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48360: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:43:58.509743 27395 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:43:58.489205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37871 (local address 127.0.0.1:48360)
0809 22:43:58.489877 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:58.489893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:58.490017 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:58.490487 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:58.490491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:58.490500 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:43:58.490750 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:58.490759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.492257 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.492261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:58.493310 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:58.493320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:58.494302 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:58.494306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:58.494436 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:58.494468 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:43:58.509232 (+ 14764us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:43:58.509456 (+ 224us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.27.190:37871: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":375,"threads_started":1}
W20260809 22:43:58.509828 27367 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48360: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 22:43:58.511341 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27291
I20260809 22:43:58.512580 27358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:43:58.610116 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27291
2026-08-09T22:43:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (777 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 22:43:58.668453 21614 mini_oidc.cc:152] Starting JWKS server
I20260809 22:43:58.670436 21614 webserver.cc:460] Webserver started at https://127.0.0.1:45785/ using document root <none> and password file <none>
I20260809 22:43:58.670652 21614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:43:58.671329 21614 webserver.cc:460] Webserver started at http://127.0.0.1:44829/ using document root <none> and password file <none>
2026-08-09T22:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:58Z Disabled control of system clock
I20260809 22:43:58.705824 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38789
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:38789
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45785/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:43:59.025656 27404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:59.025921 27404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:59.026065 27404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:59.033082 27404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:59.033190 27404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:59.033231 27404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:59.033308 27404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:59.033378 27404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:43:59.033496 27404 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:43:59.033556 27404 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45785/jwks.json
I20260809 22:43:59.042505 27404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:38789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45785/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:59.043815 27404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:59.045578 27404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:59.057250 27410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:59.057205 27411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:59.057271 27413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:59.058161 27404 server_base.cc:1034] running on GCE node
I20260809 22:43:59.058902 27404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:59.060325 27404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:59.061520 27404 hybrid_clock.cc:648] HybridClock initialized: now 1786315439061463 us; error 50 us; skew 500 ppm
I20260809 22:43:59.061866 27404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:59.064118 27404 webserver.cc:460] Webserver started at http://127.21.27.190:41345/ using document root <none> and password file <none>
I20260809 22:43:59.064821 27404 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:59.064939 27404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:59.065289 27404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:59.068221 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "6363ad4a8c264d32bac801eba1116029"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "11fa763c38360e3dc906f5131efd640a"
server_key_iv: "07a38b81393bd6018e16725676325920"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.069012 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "6363ad4a8c264d32bac801eba1116029"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "11fa763c38360e3dc906f5131efd640a"
server_key_iv: "07a38b81393bd6018e16725676325920"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.074216 27404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:43:59.077601 27419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:59.078874 27404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:43:59.079063 27404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "6363ad4a8c264d32bac801eba1116029"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "11fa763c38360e3dc906f5131efd640a"
server_key_iv: "07a38b81393bd6018e16725676325920"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.079332 27404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:59.097709 27404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:59.098510 27404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:59.098826 27404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:59.117753 27404 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:38789
I20260809 22:43:59.117774 27478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:38789 every 8 connection(s)
I20260809 22:43:59.119107 27404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:59.122763 27479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:59.127712 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27404
I20260809 22:43:59.127945 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:59.128769 21614 external_mini_cluster.cc:1428] Setting key 3bd05c16121c2417e32cdf3934d74e20
I20260809 22:43:59.134191 27479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029: Bootstrap starting.
I20260809 22:43:59.137794 27479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:59.138950 27479 log.cc:824] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:59.141748 27479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029: No bootstrap required, opened a new log
I20260809 22:43:59.145998 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:59.130663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.21.27.190:38789)
0809 22:43:59.131704 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:59.131714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:59.131753 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:43:59.133398 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:59.133418 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:59.133829 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:59.134146 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:59.134775 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.136130 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.138038 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.138707 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.138832 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.143723 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:59.143731 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:59.143744 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:59.144299 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:59.144379 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:59.144399 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:59.144492 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:59.144737 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 22:43:59.145068 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":429,"threads_started":1}
I20260809 22:43:59.149718 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:43:59.150184 27479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } }
I20260809 22:43:59.150583 27479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:59.150693 27479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6363ad4a8c264d32bac801eba1116029, State: Initialized, Role: FOLLOWER
I20260809 22:43:59.151361 27479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } }
I20260809 22:43:59.151628 27479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:59.151763 27479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:59.151922 27479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:59.154331 27479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } }
I20260809 22:43:59.154770 27479 leader_election.cc:302] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6363ad4a8c264d32bac801eba1116029; no voters:
I20260809 22:43:59.155333 27479 leader_election.cc:288] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:59.155514 27484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:59.156322 27484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [term 1 LEADER]: Becoming Leader. State: Replica: 6363ad4a8c264d32bac801eba1116029, State: Running, Role: LEADER
I20260809 22:43:59.156845 27484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } }
I20260809 22:43:59.157444 27479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:59.159741 27486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6363ad4a8c264d32bac801eba1116029. Latest consensus state: current_term: 1 leader_uuid: "6363ad4a8c264d32bac801eba1116029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } } }
I20260809 22:43:59.160424 27485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6363ad4a8c264d32bac801eba1116029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6363ad4a8c264d32bac801eba1116029" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38789 } } }
I20260809 22:43:59.160748 27485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:59.160748 27486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029 [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:59.165056 27495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:59.169401 27495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:59.177695 27495 catalog_manager.cc:1347] Generated new cluster ID: 3bda7e00984447d59e65549b713f675a
I20260809 22:43:59.177804 27495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:59.189714 27495 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:59.190867 27495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:59.199414 27495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6363ad4a8c264d32bac801eba1116029: Generated new TSK 0
I20260809 22:43:59.200188 27495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:43:59.222658 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:59.213963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.21.27.190:38789)
0809 22:43:59.214204 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:59.214211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:59.214601 (+ 390us) server_negotiation.cc:407] Connection header received
0809 22:43:59.214795 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:59.214801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:59.214900 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:59.215079 (+ 179us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:43:59.215565 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.216725 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.218145 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.218866 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.218997 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.219546 (+ 549us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:43:59.221983 (+ 2437us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45785/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:43:59.222346 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45785/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 22:43:59.222805 27482 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59520: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45785/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 22:43:59.224357 27511 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:43:59.213835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:38789 (local address 127.0.0.1:59520)
0809 22:43:59.214497 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:59.214513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:59.214630 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:59.215079 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:59.215083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:59.215092 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:43:59.215414 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:59.215422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.216942 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.216946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:59.218003 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:59.218013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.219041 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.219046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:59.219237 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.219272 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:43:59.222207 (+ 2935us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45785/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:43:59.223931 (+ 1724us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.27.190:38789: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45785/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":1415,"thread_start_us":358,"threads_started":1}
I20260809 22:43:59.225613 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27404
I20260809 22:43:59.226831 27474 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 22:43:59.235001 27512 webserver.cc:533] Webserver: sslize failed:
E20260809 22:43:59.235126 27512 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:43:59.331202 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27404
2026-08-09T22:43:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (723 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T22:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:43:59Z Disabled control of system clock
I20260809 22:43:59.424297 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:36983
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:36983
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/testchainca.cert with env {}
W20260809 22:43:59.753247 27517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:43:59.753536 27517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:43:59.753623 27517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:43:59.760691 27517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:43:59.760812 27517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:43:59.760850 27517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:43:59.760913 27517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:43:59.760952 27517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:43:59.761013 27517 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:43:59.761052 27517 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 22:43:59.770593 27517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:36983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:36983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:43:59.771816 27517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:43:59.772996 27517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:43:59.785231 27522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:59.785312 27525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:43:59.785286 27523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:43:59.786044 27517 server_base.cc:1034] running on GCE node
I20260809 22:43:59.786841 27517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:43:59.788249 27517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:43:59.789479 27517 hybrid_clock.cc:648] HybridClock initialized: now 1786315439789440 us; error 51 us; skew 500 ppm
I20260809 22:43:59.789832 27517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:43:59.792155 27517 webserver.cc:460] Webserver started at http://127.21.27.190:35095/ using document root <none> and password file <none>
I20260809 22:43:59.792903 27517 fs_manager.cc:359] Metadata directory not provided
I20260809 22:43:59.793043 27517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:43:59.793357 27517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:43:59.796253 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "b4ebbf5e4c8f46c587209c99c3b1573e"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "eaa811ed2bd94ccb9eda45b4445e7ae9"
server_key_iv: "26360e1727af8fc10c4493a73e4febdd"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.797081 27517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "b4ebbf5e4c8f46c587209c99c3b1573e"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "eaa811ed2bd94ccb9eda45b4445e7ae9"
server_key_iv: "26360e1727af8fc10c4493a73e4febdd"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.802156 27517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:43:59.805580 27533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:43:59.806890 27517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:43:59.807087 27517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "b4ebbf5e4c8f46c587209c99c3b1573e"
format_stamp: "Formatted at 2026-08-09 22:43:59 on dist-test-slave-chsk"
server_key: "eaa811ed2bd94ccb9eda45b4445e7ae9"
server_key_iv: "26360e1727af8fc10c4493a73e4febdd"
server_key_version: "encryptionkey@0"
I20260809 22:43:59.807397 27517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:43:59.835003 27517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:43:59.835987 27517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:43:59.836280 27517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:43:59.855077 27517 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:36983
I20260809 22:43:59.855103 27587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:36983 every 8 connection(s)
I20260809 22:43:59.856520 27517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:43:59.860203 27588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:43:59.866773 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27517
I20260809 22:43:59.866981 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:43:59.867723 21614 external_mini_cluster.cc:1428] Setting key c0823bc701f366e1b4f06f9e6e7450c3
I20260809 22:43:59.870493 27588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e: Bootstrap starting.
I20260809 22:43:59.873819 27588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e: Neither blocks nor log segments found. Creating new log.
I20260809 22:43:59.874883 27588 log.cc:824] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:43:59.877970 27588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e: No bootstrap required, opened a new log
I20260809 22:43:59.881410 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:59.869604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.21.27.190:36983)
0809 22:43:59.870587 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:59.870605 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:59.870690 (+ 85us) server_negotiation.cc:407] Connection header received
0809 22:43:59.872408 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:59.872429 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:59.872860 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:59.873161 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:43:59.873822 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.875275 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.876941 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.877592 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.877738 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.879001 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:43:59.879008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:43:59.879020 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:43:59.879637 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:43:59.879729 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:43:59.879750 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:43:59.879840 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:43:59.880083 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 22:43:59.880440 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":363,"threads_started":1}
I20260809 22:43:59.885404 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:43:59.886809 27588 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } }
I20260809 22:43:59.887250 27588 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:43:59.887346 27588 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4ebbf5e4c8f46c587209c99c3b1573e, State: Initialized, Role: FOLLOWER
I20260809 22:43:59.887976 27588 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } }
I20260809 22:43:59.888289 27588 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:43:59.888476 27588 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:43:59.888675 27588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:43:59.891090 27588 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } }
I20260809 22:43:59.891695 27588 leader_election.cc:302] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4ebbf5e4c8f46c587209c99c3b1573e; no voters:
I20260809 22:43:59.892294 27588 leader_election.cc:288] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:43:59.892473 27595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:43:59.893318 27595 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [term 1 LEADER]: Becoming Leader. State: Replica: b4ebbf5e4c8f46c587209c99c3b1573e, State: Running, Role: LEADER
I20260809 22:43:59.893841 27595 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } }
I20260809 22:43:59.894487 27588 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:43:59.896504 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } } }
I20260809 22:43:59.896849 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:59.897794 27600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:43:59.897646 27597 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4ebbf5e4c8f46c587209c99c3b1573e. Latest consensus state: current_term: 1 leader_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4ebbf5e4c8f46c587209c99c3b1573e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 36983 } } }
I20260809 22:43:59.898872 27597 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e [sys.catalog]: This master's current role is: LEADER
I20260809 22:43:59.903241 27600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:43:59.914535 27600 catalog_manager.cc:1347] Generated new cluster ID: 2b929a4f73464ae5b61a7dff63769b2b
I20260809 22:43:59.914660 27600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:43:59.934084 27600 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:43:59.935827 27600 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:43:59.947647 27600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4ebbf5e4c8f46c587209c99c3b1573e: Generated new TSK 0
I20260809 22:43:59.948693 27600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:44:00.017153 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:43:59.970109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.21.27.190:36983)
0809 22:43:59.970332 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:43:59.970338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:43:59.970794 (+ 456us) server_negotiation.cc:407] Connection header received
0809 22:43:59.970984 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:43:59.970992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:43:59.971079 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:43:59.971273 (+ 194us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:43:59.971680 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.972822 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.974205 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.974936 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.975074 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.975589 (+ 515us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:44:00.016287 (+ 40698us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:44:00.016779 (+ 492us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35496: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 22:44:00.017333 27592 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35496: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 22:44:00.017407 27620 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:43:59.970061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:36983 (local address 127.0.0.1:35496)
0809 22:43:59.970698 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 22:43:59.970715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:43:59.970832 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:43:59.971282 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:43:59.971286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:43:59.971295 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:43:59.971539 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:59.971546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.972955 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.972958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:59.974055 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:43:59.974066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:43:59.975094 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:43:59.975100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:43:59.975281 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:43:59.975314 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:44:00.016615 (+ 41301us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:44:00.016926 (+ 311us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.27.190:36983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":337,"threads_started":1}
I20260809 22:44:00.019255 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27517
I20260809 22:44:00.020463 27583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:00.117663 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27517
2026-08-09T22:44:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (764 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (423 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (421 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:01 dist-test-slave-chsk krb5kdc[27636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:01 dist-test-slave-chsk krb5kdc[27636](info): set up 2 sockets
Aug 09 22:44:01 dist-test-slave-chsk krb5kdc[27636](info): commencing operation
krb5kdc: starting...
W20260809 22:44:02.120565 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:02 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:44:02.362557 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39387
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:39387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:02.692757 27652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:02.693121 27652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:02.693282 27652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:02.700685 27652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:02.700836 27652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:02.700953 27652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:02.701052 27652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:02.701134 27652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:02.711397 27652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:39387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:02.712802 27652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:02.714200 27652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:02.726394 27658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:02.726372 27660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:02.726557 27662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:02.727190 27652 server_base.cc:1034] running on GCE node
I20260809 22:44:02.728050 27652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:02.729574 27652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:02.730778 27652 hybrid_clock.cc:648] HybridClock initialized: now 1786315442730743 us; error 34 us; skew 500 ppm
I20260809 22:44:02.731132 27652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:02 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315442, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:02.734400 27652 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:44:02.735973 27652 webserver.cc:460] Webserver started at http://127.21.27.190:44117/ using document root <none> and password file <none>
I20260809 22:44:02.736689 27652 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:02.736840 27652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:02.737171 27652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:02.740471 27652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "c97f4553ccc74a6fa18eaecf97196123"
format_stamp: "Formatted at 2026-08-09 22:44:02 on dist-test-slave-chsk"
server_key: "d794022b66b961d70cbb7a779e5a1261"
server_key_iv: "87a558e8f130bd96a379ccc73215cf87"
server_key_version: "encryptionkey@0"
I20260809 22:44:02.741317 27652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "c97f4553ccc74a6fa18eaecf97196123"
format_stamp: "Formatted at 2026-08-09 22:44:02 on dist-test-slave-chsk"
server_key: "d794022b66b961d70cbb7a779e5a1261"
server_key_iv: "87a558e8f130bd96a379ccc73215cf87"
server_key_version: "encryptionkey@0"
I20260809 22:44:02.746795 27652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:44:02.750465 27669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:02.751905 27652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:02.752105 27652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "c97f4553ccc74a6fa18eaecf97196123"
format_stamp: "Formatted at 2026-08-09 22:44:02 on dist-test-slave-chsk"
server_key: "d794022b66b961d70cbb7a779e5a1261"
server_key_iv: "87a558e8f130bd96a379ccc73215cf87"
server_key_version: "encryptionkey@0"
I20260809 22:44:02.752327 27652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:02.770417 27652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:02.774873 27652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:02.775233 27652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:02.795759 27652 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:39387
I20260809 22:44:02.795849 27724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:39387 every 8 connection(s)
I20260809 22:44:02.797209 27652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:02.800982 27725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:02.806208 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27652
I20260809 22:44:02.806443 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:02.807134 21614 external_mini_cluster.cc:1428] Setting key fdbe28014c934bfd2691505db470384b
I20260809 22:44:02.812335 27725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123: Bootstrap starting.
I20260809 22:44:02.816228 27725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:02.817548 27725 log.cc:824] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:02.821396 27725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123: No bootstrap required, opened a new log
Aug 09 22:44:02 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:02.832522 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:02.809290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.21.27.190:39387)
0809 22:44:02.810206 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:02.810218 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:02.810267 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:44:02.812106 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:02.812131 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:02.812577 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:02.812911 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:02.814869 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:02.816243 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:02.818112 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:02.818770 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:02.818909 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:02.823432 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:02.823442 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:02.823461 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:02.823516 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:02.827672 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:02.828425 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:02.828433 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:02.828445 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:02.828670 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:02.829043 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:02.829048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:02.829052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:02.829839 (+ 787us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:02.830172 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:02.830509 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 22:44:02.831030 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":457,"threads_started":1}
I20260809 22:44:02.832679 27725 raft_consensus.cc:348] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } }
I20260809 22:44:02.833148 27725 raft_consensus.cc:374] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:02.833289 27725 raft_consensus.cc:729] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c97f4553ccc74a6fa18eaecf97196123, State: Initialized, Role: FOLLOWER
I20260809 22:44:02.833959 27725 consensus_queue.cc:260] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } }
I20260809 22:44:02.834273 27725 raft_consensus.cc:388] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:02.834438 27725 raft_consensus.cc:482] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:02.834625 27725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:02.837203 27725 raft_consensus.cc:504] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } }
I20260809 22:44:02.837698 27725 leader_election.cc:302] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c97f4553ccc74a6fa18eaecf97196123; no voters:
I20260809 22:44:02.838291 27725 leader_election.cc:288] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:02.838449 27730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:02.839207 27730 raft_consensus.cc:686] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [term 1 LEADER]: Becoming Leader. State: Replica: c97f4553ccc74a6fa18eaecf97196123, State: Running, Role: LEADER
I20260809 22:44:02.839722 27730 consensus_queue.cc:237] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } }
I20260809 22:44:02.840449 27725 sys_catalog.cc:564] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:02.842824 27733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c97f4553ccc74a6fa18eaecf97196123. Latest consensus state: current_term: 1 leader_uuid: "c97f4553ccc74a6fa18eaecf97196123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } } }
I20260809 22:44:02.842926 27732 sys_catalog.cc:455] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c97f4553ccc74a6fa18eaecf97196123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c97f4553ccc74a6fa18eaecf97196123" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39387 } } }
I20260809 22:44:02.843278 27733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:02.843364 27732 sys_catalog.cc:458] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:02.844309 27736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:02.851079 27736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:02.860594 27736 catalog_manager.cc:1347] Generated new cluster ID: 88f5421d21db4805a26a9cfd916b9e64
I20260809 22:44:02.860738 27736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:02.884720 27736 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:02.886775 27736 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:02.894068 27736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c97f4553ccc74a6fa18eaecf97196123: Generated new TSK 0
I20260809 22:44:02.894963 27736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:44:02.957131 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:03.295728 27752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:03.296065 27752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:03.296242 27752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:03.305650 27752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:03.305845 27752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:03.306041 27752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:03.316367 27752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:03.318154 27752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:03.319717 27752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:03.333873 27757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:03.333853 27758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:03.334445 27760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:03.334813 27752 server_base.cc:1034] running on GCE node
I20260809 22:44:03.335536 27752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:03.336562 27752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:03.337808 27752 hybrid_clock.cc:648] HybridClock initialized: now 1786315443337753 us; error 50 us; skew 500 ppm
I20260809 22:44:03.338161 27752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:03 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315443, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:03.349642 27752 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:44:03.351490 27752 webserver.cc:460] Webserver started at http://127.21.27.129:44345/ using document root <none> and password file <none>
I20260809 22:44:03.352262 27752 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:03.352422 27752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:03.352761 27752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:03.355916 27752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "4250fa9b3d024eb9bb82326892033ee9"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "e6ac5869eb8a2e0712c7740f2bef1a7e"
server_key_iv: "56b9ea28d89f03e3e9417a2d1a3e37ce"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.356896 27752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "4250fa9b3d024eb9bb82326892033ee9"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "e6ac5869eb8a2e0712c7740f2bef1a7e"
server_key_iv: "56b9ea28d89f03e3e9417a2d1a3e37ce"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.362502 27752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:44:03.366354 27768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.368090 27752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:03.368309 27752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "4250fa9b3d024eb9bb82326892033ee9"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "e6ac5869eb8a2e0712c7740f2bef1a7e"
server_key_iv: "56b9ea28d89f03e3e9417a2d1a3e37ce"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.368546 27752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:03.387583 27752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:03.392587 27752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:03.392966 27752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:03.394169 27752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:03.395735 27752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:03.395849 27752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.396006 27752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:03.396082 27752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.433334 27752 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:40757
I20260809 22:44:03.433380 27890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:40757 every 8 connection(s)
I20260809 22:44:03.434824 27752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:03.444427 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27752
I20260809 22:44:03.444676 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:03.445461 21614 external_mini_cluster.cc:1428] Setting key cc867243c1a0042d38ed5e2501c53054
Aug 09 22:44:03 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315443, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:03.456902 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:03.437757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:46887 (local address 127.21.27.190:39387)
0809 22:44:03.438260 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:03.438265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:03.439189 (+ 924us) server_negotiation.cc:407] Connection header received
0809 22:44:03.441121 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:03.441127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:03.441234 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:03.441360 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:03.443054 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.443888 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.445783 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.446476 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.446624 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:03.450410 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:03.450417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:03.450421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:03.450465 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:03.453275 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:03.454095 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:03.454103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:03.454107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:03.454253 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:03.454717 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:03.454724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:03.454728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:03.455016 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:03.455210 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:03.456106 (+ 896us) server_negotiation.cc:299] Negotiation successful
0809 22:44:03.456376 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":266,"threads_started":1}
I20260809 22:44:03.457937 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:03.438191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39387 (local address 127.21.27.129:46887)
0809 22:44:03.438960 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:03.438999 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:03.440813 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:03.441615 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:03.441624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:03.442233 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:03.442822 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:03.442883 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.444066 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.444070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:03.445617 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:03.445634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.446739 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.446750 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:03.447024 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:03.447818 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:03.447848 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:03.450142 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:03.453447 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:03.453457 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:03.453476 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:03.453961 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:03.454374 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:03.454379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:03.454382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:03.454591 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:03.455185 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:03.455195 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:03.455702 (+ 507us) client_negotiation.cc:769] Sending connection context
0809 22:44:03.456003 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 22:44:03.456615 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":319,"threads_started":1}
I20260809 22:44:03.459702 27891 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39387
I20260809 22:44:03.460028 27891 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:03.460800 27891 heartbeater.cc:511] Master 127.21.27.190:39387 requested a full tablet report, sending...
I20260809 22:44:03.463045 27688 ts_manager.cc:194] Registered new tserver with Master: 4250fa9b3d024eb9bb82326892033ee9 (127.21.27.129:40757)
I20260809 22:44:03.464586 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:46887
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:44:03.502405 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:03.839915 27899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:03.840190 27899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:03.840345 27899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:03.847914 27899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:03.848027 27899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:03.848206 27899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:03.858405 27899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:03.859776 27899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:03.861094 27899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:03.874523 27904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:03.874549 27907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:03.874549 27905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:03.875515 27899 server_base.cc:1034] running on GCE node
I20260809 22:44:03.876125 27899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:03.877086 27899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:03.878322 27899 hybrid_clock.cc:648] HybridClock initialized: now 1786315443878281 us; error 35 us; skew 500 ppm
I20260809 22:44:03.878643 27899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:03 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315443, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:03.881808 27899 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:44:03.883339 27899 webserver.cc:460] Webserver started at http://127.21.27.130:46479/ using document root <none> and password file <none>
I20260809 22:44:03.883955 27899 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:03.884056 27899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:03.884377 27899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:03.887485 27899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "a1ef1f339302415f90799de9367b602e"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "d3d89ed38a50d7b721c5742850b077fd"
server_key_iv: "283102eca1ea9ab031f7cc1aee4215bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.888242 27899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "a1ef1f339302415f90799de9367b602e"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "d3d89ed38a50d7b721c5742850b077fd"
server_key_iv: "283102eca1ea9ab031f7cc1aee4215bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.893261 27899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:44:03.896766 27914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.898136 27899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:44:03.898284 27899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "a1ef1f339302415f90799de9367b602e"
format_stamp: "Formatted at 2026-08-09 22:44:03 on dist-test-slave-chsk"
server_key: "d3d89ed38a50d7b721c5742850b077fd"
server_key_iv: "283102eca1ea9ab031f7cc1aee4215bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:03.898453 27899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:03.913902 27899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:03.918236 27899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:03.918489 27899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:03.919409 27899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:03.920655 27899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:03.920728 27899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.920809 27899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:03.920866 27899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:03.955691 27899 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:34173
I20260809 22:44:03.955883 28035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:34173 every 8 connection(s)
I20260809 22:44:03.957168 27899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:03.966753 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 27899
I20260809 22:44:03.966943 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:03.967545 21614 external_mini_cluster.cc:1428] Setting key f9f2b4f9a07afd9d0bef5e027a9a5dd7
Aug 09 22:44:03 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315443, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:03.979034 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:03.959937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:58623 (local address 127.21.27.190:39387)
0809 22:44:03.960388 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:03.960393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:03.961385 (+ 992us) server_negotiation.cc:407] Connection header received
0809 22:44:03.963183 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:03.963191 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:03.963284 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:03.963386 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:03.964833 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.965589 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.967658 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.968166 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.968338 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:03.972997 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:03.973002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:03.973005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:03.973037 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:03.975925 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:03.976667 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:03.976673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:03.976677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:03.976816 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:03.977269 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:03.977275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:03.977278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:03.977548 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:03.977705 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:03.978418 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 22:44:03.978646 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":229,"threads_started":1}
I20260809 22:44:03.980093 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:03.960515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39387 (local address 127.21.27.130:58623)
0809 22:44:03.961220 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:03.961263 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:03.962868 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:03.963586 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:03.963593 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:03.964108 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:03.964662 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:03.964680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.965723 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.965727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:03.967460 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:03.967475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:03.968484 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:03.968492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:03.968761 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:03.969525 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:03.969548 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:03.972754 (+ 3206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:03.976076 (+ 3322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:03.976085 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:03.976103 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:03.976528 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:03.976937 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:03.976942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:03.976946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:03.977151 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:03.977691 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:03.977699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:03.978058 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 22:44:03.978329 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 22:44:03.978883 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":276,"threads_started":1}
I20260809 22:44:03.981644 28036 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39387
I20260809 22:44:03.981976 28036 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:03.982870 28036 heartbeater.cc:511] Master 127.21.27.190:39387 requested a full tablet report, sending...
I20260809 22:44:03.984737 27688 ts_manager.cc:194] Registered new tserver with Master: a1ef1f339302415f90799de9367b602e (127.21.27.130:34173)
I20260809 22:44:03.986047 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:58623
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:44:04.024744 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:36529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:04.353430 28042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:04.353711 28042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:04.353811 28042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:04.361296 28042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:04.361428 28042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:04.361541 28042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:04.371256 28042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39387
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:04.372538 28042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:04.373834 28042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:04.387564 28048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:04.387601 28051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:04.387672 28049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:04.388370 28042 server_base.cc:1034] running on GCE node
I20260809 22:44:04.389082 28042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:04.390045 28042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:04.391245 28042 hybrid_clock.cc:648] HybridClock initialized: now 1786315444391181 us; error 58 us; skew 500 ppm
I20260809 22:44:04.391575 28042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.394829 28042 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:44:04.396427 28042 webserver.cc:460] Webserver started at http://127.21.27.131:37391/ using document root <none> and password file <none>
I20260809 22:44:04.397135 28042 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:04.397321 28042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:04.397641 28042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:04.400795 28042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "0465827f682c429092481222a1051ff2"
format_stamp: "Formatted at 2026-08-09 22:44:04 on dist-test-slave-chsk"
server_key: "7783bca0f243a9cbb0f0bbfd59aeec07"
server_key_iv: "b43f10e146351a4126e00b549ab75c91"
server_key_version: "encryptionkey@0"
I20260809 22:44:04.401672 28042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "0465827f682c429092481222a1051ff2"
format_stamp: "Formatted at 2026-08-09 22:44:04 on dist-test-slave-chsk"
server_key: "7783bca0f243a9cbb0f0bbfd59aeec07"
server_key_iv: "b43f10e146351a4126e00b549ab75c91"
server_key_version: "encryptionkey@0"
I20260809 22:44:04.407063 28042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:44:04.410629 28059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:04.412024 28042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:04.412216 28042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "0465827f682c429092481222a1051ff2"
format_stamp: "Formatted at 2026-08-09 22:44:04 on dist-test-slave-chsk"
server_key: "7783bca0f243a9cbb0f0bbfd59aeec07"
server_key_iv: "b43f10e146351a4126e00b549ab75c91"
server_key_version: "encryptionkey@0"
I20260809 22:44:04.412436 28042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:04.427361 28042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:04.431525 28042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:04.431829 28042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:04.432971 28042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:04.434398 28042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:04.434507 28042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:04.434669 28042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:04.434746 28042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:04.467878 27891 heartbeater.cc:503] Master 127.21.27.190:39387 was elected leader, sending a full tablet report...
I20260809 22:44:04.470571 28042 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:34789
I20260809 22:44:04.470623 28183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:34789 every 8 connection(s)
I20260809 22:44:04.471968 28042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:04.479005 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28042
I20260809 22:44:04.479230 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:04.479696 21614 external_mini_cluster.cc:1428] Setting key 5da9968ad86983e19ada91d77384c62d
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:04.493541 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.474553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:39201 (local address 127.21.27.190:39387)
0809 22:44:04.474728 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.474733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.476049 (+ 1316us) server_negotiation.cc:407] Connection header received
0809 22:44:04.477860 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.477866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.477954 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.478078 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.479741 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.480725 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.483446 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.484143 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.484231 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.488278 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.488282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.488285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.488318 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.490792 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.491564 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.491568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.491570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.491699 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.492088 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.492091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.492093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.492404 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.492525 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.493132 (+ 607us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.493312 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:44:04.494817 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.475074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39387 (local address 127.21.27.131:39201)
0809 22:44:04.475888 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:04.475925 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:04.477594 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:04.478334 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:04.478342 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:04.478867 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:04.479563 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:04.479581 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.481249 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.481254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:04.483262 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:04.483279 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.484454 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.484461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:04.484808 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.485598 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:04.485632 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:04.488061 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:04.490914 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:04.490921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:04.490933 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:04.491456 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:04.491789 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:04.491792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:04.491794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:04.491992 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:04.492505 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:04.492511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:04.492809 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 22:44:04.493033 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 22:44:04.493422 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":312,"threads_started":1}
I20260809 22:44:04.496250 28184 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39387
I20260809 22:44:04.496557 28184 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:04.497272 28184 heartbeater.cc:511] Master 127.21.27.190:39387 requested a full tablet report, sending...
I20260809 22:44:04.498612 27688 ts_manager.cc:194] Registered new tserver with Master: 0465827f682c429092481222a1051ff2 (127.21.27.131:34789)
I20260809 22:44:04.499454 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:39201
I20260809 22:44:04.507934 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/test-admin.keytab.
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.539117 21614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 22:44:04.544466 21614 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc' of type FILE
W20260809 22:44:04.544516 28192 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:04.541902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.21.27.129:40757)
0809 22:44:04.542620 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.542627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.542656 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:44:04.542720 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.542727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.542945 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.543096 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.544048 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46368: BlockingRecv error: recv got EOF from 127.0.0.1:46368 (error 108)
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":390,"threads_started":1}
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.546304 21614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.547680 21614 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:44:04.563329 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.549321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.21.27.129:40757)
0809 22:44:04.549532 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.549536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.549638 (+ 102us) server_negotiation.cc:407] Connection header received
0809 22:44:04.549774 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.549777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.549837 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.549919 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.551033 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.552068 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.553385 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.553925 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.554033 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.557899 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.557911 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.557926 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.557958 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.560779 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.561385 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.561391 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.561393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.561506 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.561821 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.561824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.561826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.562194 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.562406 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.562678 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.562954 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 22:44:04.569640 28192 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:04.567747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.21.27.129:40757)
0809 22:44:04.567967 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.567974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.568228 (+ 254us) server_negotiation.cc:407] Connection header received
0809 22:44:04.568396 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.568401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.568481 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.568575 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.569415 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46400: BlockingRecv error: recv got EOF from 127.0.0.1:46400 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 22:44:04.569871 21614 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.571456 21614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.572788 21614 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:44:04.586077 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.574126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.21.27.129:40757)
0809 22:44:04.574341 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.574347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.574549 (+ 202us) server_negotiation.cc:407] Connection header received
0809 22:44:04.574656 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.574659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.574716 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.574795 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.575705 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.576375 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.577500 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.577914 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.578013 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.581417 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.581422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.581425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.581464 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.583919 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.584523 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.584526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.584528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.584652 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.584981 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.584984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.584986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.585243 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.585454 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.585694 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.585855 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 22:44:04.590505 28192 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:04.588817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46416 (local address 127.21.27.129:40757)
0809 22:44:04.589012 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.589018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.589181 (+ 163us) server_negotiation.cc:407] Connection header received
0809 22:44:04.589367 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.589372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.589455 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.589557 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.590259 (+ 702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46416: BlockingRecv error: recv got EOF from 127.0.0.1:46416 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:44:04.590665 21614 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786315394803050-21614-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.592164 21614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:04.593472 21614 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:44:04.607239 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.594704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.21.27.129:40757)
0809 22:44:04.594902 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.594908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.595071 (+ 163us) server_negotiation.cc:407] Connection header received
0809 22:44:04.595261 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.595265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.595318 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.595394 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.596284 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.596964 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.598108 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.598516 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.598606 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.601973 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.601977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.601980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.602024 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.604819 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.605427 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.605430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.605432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.605581 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.605894 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.605897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.605899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.606165 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.606473 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.606620 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.606814 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:44:04.608881 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27752
I20260809 22:44:04.620431 28192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.609534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.21.27.129:40757)
0809 22:44:04.609717 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.609725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.610063 (+ 338us) server_negotiation.cc:407] Connection header received
0809 22:44:04.610192 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.610196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.610255 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.610329 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.611239 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.611872 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.612999 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.613420 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.613515 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.615584 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.615588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.615591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.615617 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.618121 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.618685 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.618688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.618691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.618821 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.619120 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.619123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.619125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.619425 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.619614 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.619875 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.620042 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:44:04.621004 27885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:04.704943 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27752
I20260809 22:44:04.723390 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27899
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:44:04.738684 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.724357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44474 (local address 127.21.27.130:34173)
0809 22:44:04.724921 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.724928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.724955 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:44:04.725017 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.725023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.725236 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.725360 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.726347 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.727421 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.728733 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.729286 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.729386 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.733255 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.733266 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.733280 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.733310 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.736151 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.736788 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.736795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.736797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.736912 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.737247 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.737250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.737252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.737572 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.737800 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.738049 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.738330 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":277,"threads_started":1}
I20260809 22:44:04.739967 28031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:04.823985 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27899
I20260809 22:44:04.841892 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28042
Aug 09 22:44:04 dist-test-slave-chsk krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:44:04.858099 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:04.842921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.21.27.131:34789)
0809 22:44:04.843621 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:04.843628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:04.843657 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:44:04.843716 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:04.843722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:04.843930 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:04.844065 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:04.845109 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.846232 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.847543 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:04.848083 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:04.848191 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:04.852136 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:04.852147 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:04.852161 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:04.852204 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:04.855099 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.856171 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.856183 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.856186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.856310 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:04.856711 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:04.856715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:04.856720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:04.857117 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:04.857360 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:04.857594 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 22:44:04.857798 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":406,"threads_started":1}
I20260809 22:44:04.859447 28178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:04.942255 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28042
I20260809 22:44:04.960287 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 27652
I20260809 22:44:04.961541 27719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:05.065138 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 27652
2026-08-09T22:44:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4107 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:05 dist-test-slave-chsk krb5kdc[28214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:05 dist-test-slave-chsk krb5kdc[28214](info): set up 2 sockets
Aug 09 22:44:05 dist-test-slave-chsk krb5kdc[28214](info): commencing operation
krb5kdc: starting...
W20260809 22:44:06.228884 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:06 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab.
Entry for principal oryx/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab.
I20260809 22:44:06.478926 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39945
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab
--principal=oryx/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:39945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:06.812394 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:06.812690 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:06.812875 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:06.820096 28228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:06.820226 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:06.820338 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:06.820436 28228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:06.820521 28228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:06.830427 28228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:39945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab
--principal=oryx/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:06.831847 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:06.833194 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:06.845131 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:06.845299 28235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:06.845324 28236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:06.846386 28228 server_base.cc:1034] running on GCE node
I20260809 22:44:06.847121 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:06.848620 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:06.849835 28228 hybrid_clock.cc:648] HybridClock initialized: now 1786315446849801 us; error 33 us; skew 500 ppm
I20260809 22:44:06.850208 28228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:06 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:06.853348 28228 init.cc:377] Logged in from keytab as oryx/127.21.27.190@KRBTEST.COM (short username oryx)
I20260809 22:44:06.854907 28228 webserver.cc:460] Webserver started at http://127.21.27.190:39173/ using document root <none> and password file <none>
I20260809 22:44:06.855716 28228 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:06.855916 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:06.856244 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:06.859355 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "6e0ad208e06c4cf1b71b3252018ee991"
format_stamp: "Formatted at 2026-08-09 22:44:06 on dist-test-slave-chsk"
server_key: "f062d92ffc2ba3ddcbee38bd51cd9f9f"
server_key_iv: "e466d64acfbdf0f3456cd3b0fb20260e"
server_key_version: "encryptionkey@0"
I20260809 22:44:06.860141 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "6e0ad208e06c4cf1b71b3252018ee991"
format_stamp: "Formatted at 2026-08-09 22:44:06 on dist-test-slave-chsk"
server_key: "f062d92ffc2ba3ddcbee38bd51cd9f9f"
server_key_iv: "e466d64acfbdf0f3456cd3b0fb20260e"
server_key_version: "encryptionkey@0"
I20260809 22:44:06.866117 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 22:44:06.869659 28245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:06.870968 28228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:06.871227 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "6e0ad208e06c4cf1b71b3252018ee991"
format_stamp: "Formatted at 2026-08-09 22:44:06 on dist-test-slave-chsk"
server_key: "f062d92ffc2ba3ddcbee38bd51cd9f9f"
server_key_iv: "e466d64acfbdf0f3456cd3b0fb20260e"
server_key_version: "encryptionkey@0"
I20260809 22:44:06.871443 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:06.887892 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:06.892043 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:06.892349 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:06.911737 28228 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:39945
I20260809 22:44:06.911818 28299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:39945 every 8 connection(s)
I20260809 22:44:06.913192 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:06.916966 28300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:06.922426 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28228
I20260809 22:44:06.922669 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:06.923448 21614 external_mini_cluster.cc:1428] Setting key da48f305d60189f7e1c412977be7b5b5
I20260809 22:44:06.928540 28300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Bootstrap starting.
I20260809 22:44:06.932552 28300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:06.933914 28300 log.cc:824] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:06.937798 28300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: No bootstrap required, opened a new log
Aug 09 22:44:06 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:06.946478 28300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } }
I20260809 22:44:06.946905 28300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:06.947064 28300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0ad208e06c4cf1b71b3252018ee991, State: Initialized, Role: FOLLOWER
I20260809 22:44:06.947919 28300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } }
I20260809 22:44:06.948210 28300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:06.948371 28300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:06.948560 28300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:06.948786 28303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:06.925733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48240 (local address 127.21.27.190:39945)
0809 22:44:06.926601 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:06.926614 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:06.926657 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:44:06.928455 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:06.928480 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:06.928949 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:06.929301 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:06.931413 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:06.932795 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:06.934615 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:06.935754 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:06.935899 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:06.940816 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:06.940825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:06.940841 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:06.940881 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:06.944298 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:06.945065 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:06.945076 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:06.945084 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:06.945272 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:06.945697 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:06.945706 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:06.945708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:06.946284 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:06.946627 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:06.947084 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:44:06.947564 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":402,"threads_started":1}
I20260809 22:44:06.951066 28300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } }
I20260809 22:44:06.951720 28300 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0ad208e06c4cf1b71b3252018ee991; no voters:
I20260809 22:44:06.952421 28300 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:06.952589 28305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:06.953295 28305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 LEADER]: Becoming Leader. State: Replica: 6e0ad208e06c4cf1b71b3252018ee991, State: Running, Role: LEADER
I20260809 22:44:06.953898 28305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } }
I20260809 22:44:06.954717 28300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:06.957436 28307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0ad208e06c4cf1b71b3252018ee991. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } }
I20260809 22:44:06.958042 28307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:06.957665 28306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } }
I20260809 22:44:06.958470 28306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:06.961738 28315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:06.966686 28315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:06.976238 28315 catalog_manager.cc:1347] Generated new cluster ID: e6f3d2d99ee04a7088b423adb0057265
I20260809 22:44:06.976389 28315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:06.996397 28315 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:06.998371 28315 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:07.013192 28315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Generated new TSK 0
I20260809 22:44:07.014123 28315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
I20260809 22:44:07.083178 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:07.415979 28329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:07.416318 28329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:07.416491 28329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:07.423895 28329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:07.424048 28329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:07.424232 28329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:07.434046 28329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:07.435437 28329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:07.436725 28329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:07.450080 28334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:07.450187 28335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:07.450125 28337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:07.456168 28329 server_base.cc:1034] running on GCE node
I20260809 22:44:07.456856 28329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:07.457888 28329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:07.459127 28329 hybrid_clock.cc:648] HybridClock initialized: now 1786315447459080 us; error 49 us; skew 500 ppm
I20260809 22:44:07.459503 28329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:07 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:07.462742 28329 init.cc:377] Logged in from keytab as oryx/127.21.27.129@KRBTEST.COM (short username oryx)
I20260809 22:44:07.464327 28329 webserver.cc:460] Webserver started at http://127.21.27.129:38025/ using document root <none> and password file <none>
I20260809 22:44:07.465116 28329 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:07.465270 28329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:07.465588 28329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:07.468699 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "ad94b7c949834fba88ef428449347086"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "7aeb142ce33c337b3fec61c3a08f3086"
server_key_iv: "492a243952562ea8d8293bcfee8e910f"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.469501 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "ad94b7c949834fba88ef428449347086"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "7aeb142ce33c337b3fec61c3a08f3086"
server_key_iv: "492a243952562ea8d8293bcfee8e910f"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.474741 28329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:44:07.478322 28344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:07.479693 28329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:07.479877 28329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "ad94b7c949834fba88ef428449347086"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "7aeb142ce33c337b3fec61c3a08f3086"
server_key_iv: "492a243952562ea8d8293bcfee8e910f"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.480094 28329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:07.500484 28329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:07.504901 28329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:07.505204 28329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:07.506196 28329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:07.507551 28329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:07.507661 28329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:07.507814 28329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:07.507887 28329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:07.542078 28329 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:42039
I20260809 22:44:07.542122 28464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:42039 every 8 connection(s)
I20260809 22:44:07.543447 28329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:07.549152 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28329
I20260809 22:44:07.549391 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:07.550295 21614 external_mini_cluster.cc:1428] Setting key 50c13e06c916195115c64be98aa51aac
Aug 09 22:44:07 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:07.567086 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:07.546018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:49387 (local address 127.21.27.190:39945)
0809 22:44:07.546499 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:07.546506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:07.547513 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 22:44:07.549683 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:07.549687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:07.549758 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:07.549860 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:07.551780 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:07.552758 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:07.555446 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:07.555895 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:07.556033 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:07.560537 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:07.560546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:07.560553 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:07.560602 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:07.563665 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:07.564637 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:07.564641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:07.564643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:07.564762 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:07.565292 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:07.565295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:07.565297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:07.565562 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:07.565683 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:07.566640 (+ 957us) server_negotiation.cc:299] Negotiation successful
0809 22:44:07.566828 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260809 22:44:07.568718 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:07.546562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:49387)
0809 22:44:07.547349 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:07.547393 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:07.549370 (+ 1977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:07.550155 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:07.550168 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:07.550922 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:07.551608 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:07.551630 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:07.552920 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:07.552926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:07.555280 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:07.555319 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:07.556150 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:07.556159 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:07.556499 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:07.557267 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:07.557302 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:07.560275 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:07.563805 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:07.563817 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:07.563836 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:07.564511 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:07.564888 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:07.564895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:07.564901 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:07.565185 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:07.565697 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:07.565707 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:07.566220 (+ 513us) client_negotiation.cc:769] Sending connection context
0809 22:44:07.566532 (+ 312us) client_negotiation.cc:240] Negotiation successful
0809 22:44:07.567122 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":272,"threads_started":1}
I20260809 22:44:07.570370 28465 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:07.570701 28465 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:07.571491 28465 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:07.573726 28263 ts_manager.cc:194] Registered new tserver with Master: ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
I20260809 22:44:07.575428 28263 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:49387
WARNING: no policy specified for oryx/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
I20260809 22:44:07.606410 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:07.931984 28472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:07.932273 28472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:07.932367 28472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:07.939713 28472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:07.939826 28472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:07.939939 28472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:07.949747 28472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:07.950901 28472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:07.952101 28472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:07.965377 28478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:07.965415 28481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:07.965306 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:07.966260 28472 server_base.cc:1034] running on GCE node
I20260809 22:44:07.966805 28472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:07.967752 28472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:07.968981 28472 hybrid_clock.cc:648] HybridClock initialized: now 1786315447968933 us; error 41 us; skew 500 ppm
I20260809 22:44:07.969290 28472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:07 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:07.972487 28472 init.cc:377] Logged in from keytab as oryx/127.21.27.130@KRBTEST.COM (short username oryx)
I20260809 22:44:07.973769 28472 webserver.cc:460] Webserver started at http://127.21.27.130:41825/ using document root <none> and password file <none>
I20260809 22:44:07.974373 28472 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:07.974471 28472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:07.974714 28472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:07.977546 28472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "58951a3020cd4913a8fb4430901fa508"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "34946459c9ce71b0a909b96a0a6ff366"
server_key_iv: "4034348fea8f1689ca74bf9527538719"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.978324 28472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "58951a3020cd4913a8fb4430901fa508"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "34946459c9ce71b0a909b96a0a6ff366"
server_key_iv: "4034348fea8f1689ca74bf9527538719"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.983343 28472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:44:07.986745 28489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:07.988056 28472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:07.988267 28472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "58951a3020cd4913a8fb4430901fa508"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "34946459c9ce71b0a909b96a0a6ff366"
server_key_iv: "4034348fea8f1689ca74bf9527538719"
server_key_version: "encryptionkey@0"
I20260809 22:44:07.988468 28472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:08.015942 28472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:08.020311 28472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:08.020618 28472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:08.021562 28472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:08.022899 28472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:08.023006 28472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:08.023195 28472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:08.023284 28472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 22:44:08.057312 28472 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:43961
I20260809 22:44:08.057371 28606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:43961 every 8 connection(s)
I20260809 22:44:08.058564 28472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:08.059592 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28472
I20260809 22:44:08.059787 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:08.060398 21614 external_mini_cluster.cc:1428] Setting key 1ebe4e73e3e45b9a832393402045d94c
Aug 09 22:44:08 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315447, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:08.080664 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:08.061520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:35195 (local address 127.21.27.190:39945)
0809 22:44:08.061711 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:08.061717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:08.063085 (+ 1368us) server_negotiation.cc:407] Connection header received
0809 22:44:08.066023 (+ 2938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:08.066028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:08.066123 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:08.066243 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:08.067842 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.068558 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.070254 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.070699 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.070834 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:08.074532 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:08.074541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:08.074547 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:08.074586 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:08.077456 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:08.078209 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:08.078214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:08.078218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:08.078355 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:08.078823 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:08.078828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:08.078833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:08.079105 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:08.079353 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:08.080057 (+ 704us) server_negotiation.cc:299] Negotiation successful
0809 22:44:08.080309 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:44:08.081624 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:08.062062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:35195)
0809 22:44:08.062894 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:08.062944 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:08.065711 (+ 2767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:08.066471 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:08.066479 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:08.067077 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:08.067663 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:08.067689 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.068730 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.068734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:08.070105 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:08.070118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.070919 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.070927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:08.071205 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:08.071904 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:08.071926 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:08.074288 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:08.077608 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:08.077617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:08.077633 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:08.078065 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:08.078471 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:08.078476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:08.078480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:08.078693 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:08.079328 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:08.079335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:08.079683 (+ 348us) client_negotiation.cc:769] Sending connection context
0809 22:44:08.079944 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 22:44:08.080484 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":313,"threads_started":1}
I20260809 22:44:08.083175 28607 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:08.083559 28607 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.21.27.131@KRBTEST.COM; defaulting to no policy
I20260809 22:44:08.084539 28607 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:08.086050 28263 ts_manager.cc:194] Registered new tserver with Master: 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961)
I20260809 22:44:08.086973 28263 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:35195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
I20260809 22:44:08.116662 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:08.442462 28613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:08.442853 28613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:08.442984 28613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:08.450424 28613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:08.450531 28613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:08.450644 28613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:08.460410 28613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:08.461611 28613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:08.462911 28613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:08.476150 28618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:08.476080 28619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:08.476017 28621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:08.477254 28613 server_base.cc:1034] running on GCE node
I20260809 22:44:08.477819 28613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:08.478694 28613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:08.479892 28613 hybrid_clock.cc:648] HybridClock initialized: now 1786315448479851 us; error 32 us; skew 500 ppm
I20260809 22:44:08.480214 28613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:08 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:08.487775 28613 init.cc:377] Logged in from keytab as oryx/127.21.27.131@KRBTEST.COM (short username oryx)
I20260809 22:44:08.489256 28613 webserver.cc:460] Webserver started at http://127.21.27.131:45915/ using document root <none> and password file <none>
I20260809 22:44:08.489887 28613 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:08.490024 28613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:08.490281 28613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:08.493083 28613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
format_stamp: "Formatted at 2026-08-09 22:44:08 on dist-test-slave-chsk"
server_key: "8469676da8d1e153e35b3316d357c245"
server_key_iv: "6892369d99e56f760d3e88a9e5453c1c"
server_key_version: "encryptionkey@0"
I20260809 22:44:08.493829 28613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
format_stamp: "Formatted at 2026-08-09 22:44:08 on dist-test-slave-chsk"
server_key: "8469676da8d1e153e35b3316d357c245"
server_key_iv: "6892369d99e56f760d3e88a9e5453c1c"
server_key_version: "encryptionkey@0"
I20260809 22:44:08.498901 28613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:44:08.502547 28628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:08.503986 28613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:08.504184 28613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
format_stamp: "Formatted at 2026-08-09 22:44:08 on dist-test-slave-chsk"
server_key: "8469676da8d1e153e35b3316d357c245"
server_key_iv: "6892369d99e56f760d3e88a9e5453c1c"
server_key_version: "encryptionkey@0"
I20260809 22:44:08.504391 28613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:08.541558 28613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:08.546010 28613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:08.546335 28613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:08.547367 28613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:08.548677 28613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:08.548797 28613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:08.548950 28613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:08.549024 28613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:08.578552 28465 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
I20260809 22:44:08.586813 28613 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45727
I20260809 22:44:08.586894 28748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45727 every 8 connection(s)
I20260809 22:44:08.588415 28613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:08.590361 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28613
I20260809 22:44:08.590548 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:08.591059 21614 external_mini_cluster.cc:1428] Setting key ae434d4782fbcb79c971193cf97de86f
Aug 09 22:44:08 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315448, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:08.609078 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:08.591132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:39143 (local address 127.21.27.190:39945)
0809 22:44:08.591655 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:08.591662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:08.592896 (+ 1234us) server_negotiation.cc:407] Connection header received
0809 22:44:08.595122 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:08.595126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:08.595253 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:08.595348 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:08.596740 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.597488 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.598936 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.599386 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.599480 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:08.602973 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:08.602978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:08.602981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:08.603012 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:08.605770 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:08.606702 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:08.606707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:08.606710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:08.606878 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:08.607398 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:08.607402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:08.607404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:08.607672 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:08.607790 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:08.608563 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 22:44:08.608746 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":243,"threads_started":1}
I20260809 22:44:08.610116 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:08.591804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:39143)
0809 22:44:08.592724 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:08.592785 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:08.594873 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:08.595545 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:08.595553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:08.596052 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:08.596585 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:08.596606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.597616 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.597619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:08.598815 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:08.598826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:08.599566 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:08.599570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:08.599765 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:08.600476 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:08.600497 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:08.602755 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:08.605917 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:08.605926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:08.605943 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:08.606568 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:08.607006 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:08.607009 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:08.607011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:08.607300 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:08.607804 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:08.607810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:08.608214 (+ 404us) client_negotiation.cc:769] Sending connection context
0809 22:44:08.608481 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 22:44:08.608938 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":375,"threads_started":1}
I20260809 22:44:08.611512 28749 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:08.611828 28749 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:08.612534 28749 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:08.614018 28263 ts_manager.cc:194] Registered new tserver with Master: c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
I20260809 22:44:08.614842 28263 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:39143
I20260809 22:44:08.618992 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.27.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.189.keytab.
Entry for principal oryx/127.21.27.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.189.keytab.
I20260809 22:44:08.672268 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:41517
--webserver_interface=127.21.27.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.189.keytab
--principal=oryx/127.21.27.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:39945,127.21.27.189:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:09.005334 28755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:09.005650 28755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:09.005856 28755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:09.013829 28755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:09.013957 28755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:09.014050 28755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:09.014153 28755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:09.014233 28755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:09.024766 28755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:39945,127.21.27.189:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.189.keytab
--principal=oryx/127.21.27.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.27.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
W20260809 22:44:09.025203 28755 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.27.190:39945,127.21.27.189:41517'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 22:44:09.026371 28755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:09.027846 28755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:09.040107 28764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.040131 28761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.040160 28760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:09.041083 28755 server_base.cc:1034] running on GCE node
I20260809 22:44:09.041850 28755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:09.043342 28755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:09.044567 28755 hybrid_clock.cc:648] HybridClock initialized: now 1786315449044506 us; error 33 us; skew 500 ppm
I20260809 22:44:09.044929 28755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:09.048341 28755 init.cc:377] Logged in from keytab as oryx/127.21.27.189@KRBTEST.COM (short username oryx)
I20260809 22:44:09.049964 28755 webserver.cc:460] Webserver started at http://127.21.27.189:41391/ using document root <none> and password file <none>
I20260809 22:44:09.050734 28755 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:09.050834 28755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:09.051362 28755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:09.054841 28755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/instance:
uuid: "cf413bf632fb435e820449fc27926bce"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "b45cfc4cde5f765e14ecefdbda23b8c2"
server_key_iv: "f5f97d0ae941fd8e94b96e02d17a458d"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.055823 28755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal/instance:
uuid: "cf413bf632fb435e820449fc27926bce"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "b45cfc4cde5f765e14ecefdbda23b8c2"
server_key_iv: "f5f97d0ae941fd8e94b96e02d17a458d"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.062152 28755 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260809 22:44:09.065930 28773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:09.067549 28755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:09.067762 28755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "cf413bf632fb435e820449fc27926bce"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "b45cfc4cde5f765e14ecefdbda23b8c2"
server_key_iv: "f5f97d0ae941fd8e94b96e02d17a458d"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.068001 28755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:09.084915 28755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:09.089569 28607 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
I20260809 22:44:09.089632 28755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:09.089944 28755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:09.109619 28755 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.189:41517
I20260809 22:44:09.109637 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.189:41517 every 8 connection(s)
I20260809 22:44:09.111109 28755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
I20260809 22:44:09.114827 28832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:09.116840 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28755
I20260809 22:44:09.124763 28832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.189@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:09.141178 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.119305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.21.27.189:41517)
0809 22:44:09.120135 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.120146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.120197 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:44:09.121782 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.121808 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.122221 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.122551 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.124467 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.125818 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.127457 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.128604 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.128796 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.133313 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.133320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.133330 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.133368 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.136869 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.137576 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.137582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.137588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.137797 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.138306 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.138309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.138312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.138934 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.139244 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.139728 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.140106 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":406,"threads_started":1}
I20260809 22:44:09.143410 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.127540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.21.27.190:39945)
0809 22:44:09.128085 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.128094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.128569 (+ 475us) server_negotiation.cc:407] Connection header received
0809 22:44:09.128813 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.128818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.128920 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.129038 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.130526 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.131348 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.132938 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.133368 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.133515 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.137135 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.137139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.137142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.137180 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.140385 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.141128 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.141139 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.141141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.141310 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.141740 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.141748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.141751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.142096 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.142266 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.142737 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.142983 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":283,"threads_started":1}
I20260809 22:44:09.143465 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.127612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43690)
0809 22:44:09.128430 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.128470 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.128650 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.129126 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.129133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.129980 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.130358 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.130375 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.131496 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.131503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.132789 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.132801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.133568 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.133575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.133776 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.134474 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.134491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.136930 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.140519 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.140535 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.140542 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.140990 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.141463 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.141466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.141469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.141608 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.142258 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.142266 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.142430 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.142655 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.142996 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":483,"threads_started":1}
I20260809 22:44:09.145330 28832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.189@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
I20260809 22:44:09.158598 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.145931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59570)
0809 22:44:09.146115 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.146131 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.146249 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.146516 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.146520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.146892 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.147018 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.147026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.148027 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.148033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.149013 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.149025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.149714 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.149719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.149883 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.150431 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.150448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.152745 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.156121 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.156126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.156130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.156529 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.156895 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.156901 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.156904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.157034 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.157586 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.157591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.157743 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.157885 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.158217 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:44:09.158660 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.146113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.21.27.189:41517)
0809 22:44:09.146264 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.146269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.146287 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:09.146335 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.146340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.146404 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.146500 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.147175 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.147881 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.149156 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.149557 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.149694 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.152943 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.152948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.152951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.152981 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.155970 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.156654 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.156662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.156665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.156785 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.157139 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.157146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.157149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.157467 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.157629 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.157971 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.158353 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":53}
I20260809 22:44:09.164076 28832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Bootstrap starting.
I20260809 22:44:09.167217 28832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:09.168300 28832 log.cc:824] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:09.171010 28832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: No bootstrap required, opened a new log
I20260809 22:44:09.178375 28832 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } }
I20260809 22:44:09.178719 28832 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:09.178820 28832 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf413bf632fb435e820449fc27926bce, State: Initialized, Role: FOLLOWER
I20260809 22:44:09.179471 28832 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } }
I20260809 22:44:09.180371 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } } }
I20260809 22:44:09.180702 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:09.181362 28832 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:44:09.190446 28851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:44:09.190644 28851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:44:09.192781 28755 master_runner.cc:428] Detected that this master cf413bf632fb435e820449fc27926bce is joining an existing cluster
I20260809 22:44:09.192911 28755 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.27.189:41517
I20260809 22:44:09.212329 28839 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:09.212738 28839 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } }
I20260809 22:44:09.214243 28839 leader_election.cc:288] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945)
I20260809 22:44:09.215440 28273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf413bf632fb435e820449fc27926bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0ad208e06c4cf1b71b3252018ee991" is_pre_election: true
I20260809 22:44:09.216822 28775 leader_election.cc:302] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: cf413bf632fb435e820449fc27926bce; no voters: 6e0ad208e06c4cf1b71b3252018ee991
I20260809 22:44:09.217588 28839 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:44:09.229576 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.201711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.21.27.190:39945)
0809 22:44:09.202200 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.202207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.202589 (+ 382us) server_negotiation.cc:407] Connection header received
0809 22:44:09.202876 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.202883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.203001 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.203086 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.205001 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.206472 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.209356 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.210072 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.210220 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.221685 (+ 11465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.221695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.221703 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.221753 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.225917 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.226902 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.226909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.226913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.227090 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.227647 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.227655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.227659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.228082 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.228220 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.228905 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.229197 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
WARNING: no policy specified for oryx/127.21.27.188@KRBTEST.COM; defaulting to no policy
I20260809 22:44:09.231029 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.201767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43698)
0809 22:44:09.202465 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.202491 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.202705 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.203352 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.203358 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.204239 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.204775 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.204791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.206719 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.206728 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.209171 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.209194 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.210308 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.210317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.210590 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.219360 (+ 8770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.219387 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.220682 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.226120 (+ 5438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.226128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.226135 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.226724 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.227302 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.227308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.227314 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.227504 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.228330 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.228338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.228578 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.230115 (+ 1537us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.230493 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":442,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.188@KRBTEST.COM" created.
I20260809 22:44:09.239552 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.203160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59574 (local address 127.21.27.189:41517)
0809 22:44:09.203360 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.203365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.221461 (+ 18096us) server_negotiation.cc:407] Connection header received
0809 22:44:09.221623 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.221629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.221695 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.221773 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.223651 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.225136 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.227125 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.227878 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.227973 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.230383 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.230390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.230395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.230443 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.236098 (+ 5655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.237077 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.237085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.237091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.237256 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.237686 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.237692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.237698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.238104 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.238267 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.238806 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.239061 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:44:09.240736 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.202697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59574)
0809 22:44:09.221314 (+ 18617us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.221335 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.221482 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.221982 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.221987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.223106 (+ 1119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.223503 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.223515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.225353 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.225358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.226981 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.226997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.228269 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.228275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.228442 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.229110 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.229133 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.230185 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.236253 (+ 6068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.236261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.236268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.236936 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.237389 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.237396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.237402 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.237575 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.238354 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.238360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.238527 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.239810 (+ 1283us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.240167 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18461,"thread_start_us":18285,"threads_started":1}
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.190@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
I20260809 22:44:09.264097 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.240049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.21.27.189:41517)
0809 22:44:09.242207 (+ 2158us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.242213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.247884 (+ 5671us) server_negotiation.cc:407] Connection header received
0809 22:44:09.248211 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.248222 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.248304 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.248423 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.249564 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.250355 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.251729 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.252226 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.252358 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.256657 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.256666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.256676 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.256715 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.260681 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.261639 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.261646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.261650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.261815 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.262342 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.262352 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.262357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.262724 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.262885 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.263472 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.263720 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2036}
I20260809 22:44:09.264086 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.239578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59586)
0809 22:44:09.247708 (+ 8130us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.247759 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.247964 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.248479 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.248487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.249168 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.249406 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.249415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.250487 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.250494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.251581 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.251593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.252408 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.252414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.252607 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.253295 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.253313 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.256415 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.260815 (+ 4400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.260826 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.260832 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.261488 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.262076 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.262079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.262082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.262219 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.262915 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.262928 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.263115 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.263335 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.263694 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7891,"thread_start_us":7745,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.188.keytab.
Entry for principal oryx/127.21.27.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.188.keytab.
I20260809 22:44:09.266185 28263 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.27.189:41517: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cf413bf632fb435e820449fc27926bce"
member_type: NON_VOTER
last_known_addr {
host: "127.21.27.189"
port: 41517
}
attrs {
promote: true
}
}
dest_uuid: "6e0ad208e06c4cf1b71b3252018ee991"
cas_config_opid_index: -1
I20260809 22:44:09.267235 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:35985
--webserver_interface=127.21.27.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.188.keytab
--principal=oryx/127.21.27.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:44:09.267132 28263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } }
I20260809 22:44:09.280035 28865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cf413bf632fb435e820449fc27926bce (127.21.27.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } }
I20260809 22:44:09.281488 28801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:09.285346 28801 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 22:44:09.284886 28865 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.27.189:41517
I20260809 22:44:09.286680 28865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:09.287326 28755 master.cc:537] Master@127.21.27.189:41517 shutting down...
I20260809 22:44:09.287330 28866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } }
I20260809 22:44:09.287844 28866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
W20260809 22:44:09.304409 28247 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 22:44:09.310108 28755 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:09.310815 28755 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:09.311163 28755 tablet_replica.cc:330] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: stopping tablet replica
I20260809 22:44:09.332795 28755 master.cc:552] Master@127.21.27.189:41517 shutdown complete.
I20260809 22:44:09.333112 28755 master_runner.cc:305] Clearing existing system tablet
I20260809 22:44:09.333830 28755 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:09.334044 28755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:09.343698 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.343111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59596)
0809 22:44:09.343346 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.343485 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:41517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 22:44:09.344247 28247 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 -> Peer cf413bf632fb435e820449fc27926bce (127.21.27.189:41517): Couldn't send request to peer cf413bf632fb435e820449fc27926bce. Status: Network error: Client connection negotiation failed: client connection to 127.21.27.189:41517: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:44:09.348421 28755 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.002s sys 0.001s
I20260809 22:44:09.351850 28873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:09.352604 28755 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:09.353084 28755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:44:09.353197 28755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "cf413bf632fb435e820449fc27926bce"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "b45cfc4cde5f765e14ecefdbda23b8c2"
server_key_iv: "f5f97d0ae941fd8e94b96e02d17a458d"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.359320 28755 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:09.366194 28755 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:44:09.366397 28755 log.cc:1196] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 22:44:09.367018 28755 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Deleting consensus metadata
I20260809 22:44:09.367354 28755 master_runner.cc:315] Copying system tablet from 127.21.27.190:39945
I20260809 22:44:09.380865 28755 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: tablet copy: Beginning tablet copy session from remote peer at address 127.21.27.190:39945
I20260809 22:44:09.399369 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.381738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.21.27.190:39945)
0809 22:44:09.382037 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.382049 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.382727 (+ 678us) server_negotiation.cc:407] Connection header received
0809 22:44:09.383077 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.383083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.383293 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.383451 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:09.385012 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.386356 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.388642 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.389351 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.389449 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.392132 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:09.392140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:09.392144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:09.392186 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:09.395846 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.396697 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.396706 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.396710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.396889 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:09.397560 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:09.397566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:09.397570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:09.397942 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:09.398071 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.398700 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.398971 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:44:09.400484 28880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.381988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43714)
0809 22:44:09.382564 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.382599 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.382855 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.383734 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.383739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.384432 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:09.384773 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.384787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.386591 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.386596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.388437 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.388451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.389781 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.389787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.389966 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.390667 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:09.390690 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:09.391901 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:09.396022 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.396031 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.396035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.396542 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.397079 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:09.397087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:09.397092 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:09.397423 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:09.398181 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:09.398188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:09.398380 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.399705 (+ 1325us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.400030 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":1162,"threads_started":1}
I20260809 22:44:09.401463 28284 tablet_copy_service.cc:140] P 6e0ad208e06c4cf1b71b3252018ee991: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cf413bf632fb435e820449fc27926bce ({username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:43714)
I20260809 22:44:09.401737 28284 tablet_copy_service.cc:161] P 6e0ad208e06c4cf1b71b3252018ee991: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cf413bf632fb435e820449fc27926bce at {username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:43714: session id = cf413bf632fb435e820449fc27926bce-00000000000000000000000000000000
I20260809 22:44:09.405823 28284 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:44:09.408087 28755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:44:09.414916 28755 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: tablet copy: Starting download of 0 data blocks...
I20260809 22:44:09.415259 28755 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: tablet copy: Starting download of 1 WAL segments...
I20260809 22:44:09.418035 28755 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:44:09.422482 28284 tablet_copy_service.cc:342] P 6e0ad208e06c4cf1b71b3252018ee991: Request end of tablet copy session cf413bf632fb435e820449fc27926bce-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:43714
I20260809 22:44:09.422674 28284 tablet_copy_service.cc:433] P 6e0ad208e06c4cf1b71b3252018ee991: ending tablet copy session cf413bf632fb435e820449fc27926bce-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cf413bf632fb435e820449fc27926bce
I20260809 22:44:09.435449 28755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:09.438330 28881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.439853 28882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.446645 28884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:09.448338 28755 server_base.cc:1034] running on GCE node
I20260809 22:44:09.448781 28755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:09.449618 28755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:09.451036 28755 hybrid_clock.cc:648] HybridClock initialized: now 1786315449450949 us; error 49 us; skew 500 ppm
I20260809 22:44:09.451297 28755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:09.453549 28755 init.cc:377] Logged in from keytab as oryx/127.21.27.189@KRBTEST.COM (short username oryx)
I20260809 22:44:09.454995 28755 webserver.cc:460] Webserver started at http://127.21.27.189:33553/ using document root <none> and password file <none>
I20260809 22:44:09.455449 28755 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:09.455539 28755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:09.458837 28755 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260809 22:44:09.460944 28892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:09.461382 28755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 22:44:09.461506 28755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "cf413bf632fb435e820449fc27926bce"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "b45cfc4cde5f765e14ecefdbda23b8c2"
server_key_iv: "f5f97d0ae941fd8e94b96e02d17a458d"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.461678 28755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:09.494555 28755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:09.498751 28755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:09.528056 28755 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.189:41517
I20260809 22:44:09.528082 28948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.189:41517 every 8 connection(s)
I20260809 22:44:09.529115 28755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
I20260809 22:44:09.536248 28949 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:44:09.538796 28949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Bootstrap starting.
I20260809 22:44:09.552768 28949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:44:09.553247 28949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Bootstrap complete.
I20260809 22:44:09.554168 28949 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } }
I20260809 22:44:09.554500 28949 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf413bf632fb435e820449fc27926bce, State: Initialized, Role: LEARNER
I20260809 22:44:09.554792 28949 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } }
I20260809 22:44:09.555652 28952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } }
I20260809 22:44:09.555970 28949 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:09.556144 28952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: LEARNER
I20260809 22:44:09.564550 28964 catalog_manager.cc:1259] Loaded cluster ID: e6f3d2d99ee04a7088b423adb0057265
I20260809 22:44:09.564678 28964 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: loading cluster ID for follower catalog manager: success
I20260809 22:44:09.567644 28964 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: acquiring CA information for follower catalog manager: success
I20260809 22:44:09.569576 28964 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:44:09.617355 28749 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
I20260809 22:44:09.708426 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.693927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59600)
0809 22:44:09.694370 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:09.694403 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:09.694641 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:09.698505 (+ 3864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:09.698511 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:09.698565 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:09.698855 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.698870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.701814 (+ 2944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.701821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.704376 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:09.704395 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.707009 (+ 2614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.707025 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:09.707481 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.707567 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 22:44:09.707692 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 22:44:09.707921 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260809 22:44:09.709163 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:09.694307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.21.27.189:41517)
0809 22:44:09.697798 (+ 3491us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:09.697814 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:09.697857 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:44:09.697978 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:09.697992 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:09.698160 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:09.698418 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:09.699084 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.701570 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.704643 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:09.706726 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:09.706882 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:09.707570 (+ 688us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:09.707756 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 22:44:09.708113 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3239,"thread_start_us":2958,"threads_started":1}
I20260809 22:44:09.711884 28922 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 22:44:09.712931 28952 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0ad208e06c4cf1b71b3252018ee991. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } }
I20260809 22:44:09.713419 28952 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: LEARNER
W20260809 22:44:09.857333 28867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:09.857764 28867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:09.858034 28867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:09.869025 28867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:09.869213 28867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:09.869328 28867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:09.869472 28867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:09.869608 28867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:09.882093 28867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.188.keytab
--principal=oryx/127.21.27.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:35985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.27.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:09.883723 28867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:09.885313 28867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:09.901135 28970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.901530 28973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:09.902500 28971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:09.903357 28867 server_base.cc:1034] running on GCE node
I20260809 22:44:09.904260 28867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:09.905892 28867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:09.907214 28867 hybrid_clock.cc:648] HybridClock initialized: now 1786315449907045 us; error 71 us; skew 500 ppm
I20260809 22:44:09.907702 28867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:09 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:09.912433 28867 init.cc:377] Logged in from keytab as oryx/127.21.27.188@KRBTEST.COM (short username oryx)
I20260809 22:44:09.923312 28867 webserver.cc:460] Webserver started at http://127.21.27.188:36149/ using document root <none> and password file <none>
I20260809 22:44:09.924427 28867 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:09.924774 28867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:09.925194 28867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:09.929490 28867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/instance:
uuid: "2d0d1f658c624c139355f60c372624d3"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "3f7f0ad6634b463a09e5e76bf8e7f034"
server_key_iv: "95ee9511561e199da1b152a6330e7954"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.930433 28867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal/instance:
uuid: "2d0d1f658c624c139355f60c372624d3"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "3f7f0ad6634b463a09e5e76bf8e7f034"
server_key_iv: "95ee9511561e199da1b152a6330e7954"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.936975 28867 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260809 22:44:09.941296 28981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:09.942672 28867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:44:09.942909 28867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "2d0d1f658c624c139355f60c372624d3"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "3f7f0ad6634b463a09e5e76bf8e7f034"
server_key_iv: "95ee9511561e199da1b152a6330e7954"
server_key_version: "encryptionkey@0"
I20260809 22:44:09.943233 28867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:09.956344 28867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:09.962218 28867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:09.962682 28867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:09.999274 28867 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.188:35985
I20260809 22:44:09.999352 29037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.188:35985 every 8 connection(s)
I20260809 22:44:10.001078 28867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
I20260809 22:44:10.003831 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 28867
I20260809 22:44:10.007441 29038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:10.018329 29038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.188@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:10.033777 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.006267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42652 (local address 127.21.27.188:35985)
0809 22:44:10.007559 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.007599 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.007679 (+ 80us) server_negotiation.cc:407] Connection header received
0809 22:44:10.010188 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.010212 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.010634 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.010993 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.013874 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.015573 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.017418 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.018213 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.018427 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.024226 (+ 5799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.024237 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.024255 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.024341 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.028512 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.029571 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.029578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.029583 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.029755 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.030266 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.030274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.030280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.031016 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.031396 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.031878 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.032332 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":681,"threads_started":1}
I20260809 22:44:10.039551 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.021482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.21.27.190:39945)
0809 22:44:10.022431 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.022437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.023347 (+ 910us) server_negotiation.cc:407] Connection header received
0809 22:44:10.023564 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.023570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.023662 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.023818 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.025723 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.026785 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.029011 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.029748 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.029884 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.033771 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.033776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.033779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.033813 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.037134 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.037783 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.037787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.037790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.037904 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.038275 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.038278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.038280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.038551 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.038664 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.039062 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.039301 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":640,"threads_started":1}
I20260809 22:44:10.039553 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.021363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43726)
0809 22:44:10.023192 (+ 1829us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.023235 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.023396 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.023851 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.023856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.025127 (+ 1271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.025512 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.025558 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.026952 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.026958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.028874 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.028884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.029933 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.029937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.030063 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.030622 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.030642 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.033579 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.037253 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.037262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.037266 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.037666 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.037993 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.037996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.037998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.038133 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.038656 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.038661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.038814 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.038957 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.039268 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"thread_start_us":1326,"threads_started":1}
I20260809 22:44:10.040990 29038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.188@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
I20260809 22:44:10.053195 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.041548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59612)
0809 22:44:10.041741 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.041757 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.041835 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.042232 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.042235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.042536 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.042664 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.042672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.043741 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.043745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.044648 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.044658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.045402 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.045406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.045525 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.046076 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.046092 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.048248 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.051059 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.051062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.051065 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.051495 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.051844 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.051847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.051849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.051968 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.052513 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.052515 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.052638 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.052747 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.052960 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 22:44:10.053233 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.041803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.21.27.189:41517)
0809 22:44:10.041967 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.041971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.041987 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:44:10.042051 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.042055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.042115 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.042194 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.042805 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.043615 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.044810 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.045267 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.045376 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.048464 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.048469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.048473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.048506 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.050936 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.051603 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.051607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.051609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.051749 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.052055 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.052058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.052060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.052409 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.052522 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.052834 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.053007 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:44:10.054352 29038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315449, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.188@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
I20260809 22:44:10.067720 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.055035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.0.0.1:42658)
0809 22:44:10.055357 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.055377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.055478 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.055773 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.055777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.056346 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.056530 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.056540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.057627 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.057635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.058467 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.058478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.059393 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.059398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.059607 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.060207 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.060223 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.062319 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.065148 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.065154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.065159 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.065525 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.065914 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.065920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.065923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.066083 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.066600 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.066604 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.066780 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.066922 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.067257 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 22:44:10.067749 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.055260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42658 (local address 127.21.27.188:35985)
0809 22:44:10.055433 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.055439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.055455 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:44:10.055594 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.055598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.055656 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.055758 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.056661 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.057514 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.058604 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.059222 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.059373 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.062501 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.062505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.062509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.062536 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.065005 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.065648 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.065655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.065659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.065788 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.066200 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.066205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.066208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.066483 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.066642 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.066998 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.067423 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":193,"server-negotiator.queue_time_us":53}
I20260809 22:44:10.072872 29038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Bootstrap starting.
I20260809 22:44:10.076332 29038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:10.077373 29038 log.cc:824] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:10.080099 29038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: No bootstrap required, opened a new log
I20260809 22:44:10.087563 29038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } }
I20260809 22:44:10.087982 29038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:10.088162 29038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0d1f658c624c139355f60c372624d3, State: Initialized, Role: FOLLOWER
I20260809 22:44:10.088815 29038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } }
I20260809 22:44:10.089733 29045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } } }
I20260809 22:44:10.090090 29045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:10.090826 29038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:44:10.104662 29056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:44:10.104888 29056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:44:10.109760 28867 master_runner.cc:428] Detected that this master 2d0d1f658c624c139355f60c372624d3 is joining an existing cluster
I20260809 22:44:10.109992 28867 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.27.188:35985
I20260809 22:44:10.140942 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28329
I20260809 22:44:10.147382 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.117190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.21.27.190:39945)
0809 22:44:10.117387 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.117395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.118940 (+ 1545us) server_negotiation.cc:407] Connection header received
0809 22:44:10.119328 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.119338 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.119456 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.119550 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.122175 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.124057 (+ 1882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.128152 (+ 4095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.129318 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.129463 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.135465 (+ 6002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.135473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.135479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.135529 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.140290 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.140980 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.140987 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.140991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.141262 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.143354 (+ 2092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.143361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.143364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.144194 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.144396 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.146432 (+ 2036us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.146780 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:44:10.148193 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.117495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43732)
0809 22:44:10.118785 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.118812 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.119092 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.119906 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.119914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.121305 (+ 1391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.121873 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.121894 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.124430 (+ 2536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.124439 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.127864 (+ 3425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.127896 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.130005 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.130015 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.130359 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.132779 (+ 2420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.132801 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.135117 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.140391 (+ 5274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.140398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.140403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.140835 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.141692 (+ 857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.141702 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.141706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.142026 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.145800 (+ 3774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.145804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.146078 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.146260 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.146829 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":877,"threads_started":1}
I20260809 22:44:10.156574 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.117844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.0.0.1:42670)
0809 22:44:10.131803 (+ 13959us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.131819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.132008 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.132761 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.132765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.134677 (+ 1912us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.135388 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.135402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.137866 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.137874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.140234 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.140254 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.142960 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.142969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.143451 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.144324 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.144348 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.145948 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.152617 (+ 6669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.152622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.152626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.153229 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.153633 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.153637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.153643 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.153848 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.155219 (+ 1371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.155225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.155478 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.155663 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.156056 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13786,"thread_start_us":14662,"threads_started":1}
I20260809 22:44:10.158727 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.117197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.21.27.188:35985)
0809 22:44:10.117527 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.117534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.131928 (+ 14394us) server_negotiation.cc:407] Connection header received
0809 22:44:10.132136 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.132141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.132283 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.132459 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.135536 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.137567 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.141010 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.142277 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.142407 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.146222 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.146229 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.146233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.146269 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.152477 (+ 6208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.153365 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.153370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.153373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.153513 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.153968 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.153973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.153982 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.154525 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.156853 (+ 2328us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.156986 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.157189 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":89}
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.129@KRBTEST.COM
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.190@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
I20260809 22:44:10.178772 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.142196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.21.27.129:42039)
0809 22:44:10.143473 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.143483 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.143515 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:44:10.143586 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.143595 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.143947 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.148843 (+ 4896us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.149371 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.151492 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.154210 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.155668 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.166111 (+ 10443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.166981 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.166999 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.167021 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.167067 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.172989 (+ 5922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.174101 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.174111 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.174114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.174325 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.174869 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.174874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.174878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.175408 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.177398 (+ 1990us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.177645 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.178093 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":734,"threads_started":1}
I20260809 22:44:10.180826 28460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:10.194536 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.119470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.21.27.189:41517)
0809 22:44:10.119736 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.119742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.134438 (+ 14696us) server_negotiation.cc:407] Connection header received
0809 22:44:10.134790 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.134808 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.134900 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.134990 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.138075 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.139410 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.147048 (+ 7638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.148222 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.148336 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.181526 (+ 33190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.181538 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.181557 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.181622 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.187447 (+ 5825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.188874 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.188884 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.188890 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.189137 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.189831 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.189839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.189844 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.190394 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.190562 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.193267 (+ 2705us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.193795 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 22:44:10.206399 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.118940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59618)
0809 22:44:10.134305 (+ 15365us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.134328 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.134583 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.135304 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.135310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.136924 (+ 1614us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.137841 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.137855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.139646 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.139651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.146845 (+ 7194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.146859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.150159 (+ 3300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.150171 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.165633 (+ 15462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.169118 (+ 3485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.169158 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.172941 (+ 3783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.187754 (+ 14813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.187763 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.187770 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.188667 (+ 897us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.189341 (+ 674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.189348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.189354 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.189660 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.190903 (+ 1243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.190911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.192712 (+ 1801us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.194917 (+ 2205us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.195431 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15062,"mutex_wait_us":138,"thread_start_us":12357,"threads_started":1}
I20260809 22:44:10.210345 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.157805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.0.0.1:42686)
0809 22:44:10.158187 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.158212 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.158343 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.161652 (+ 3309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.161661 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.162469 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.162952 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.162964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.166515 (+ 3551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.166525 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.168672 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.168698 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.173237 (+ 4539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.173247 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.173583 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.174513 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.174560 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.179065 (+ 4505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.206454 (+ 27389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.206465 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.206472 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.207255 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.207921 (+ 666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.207927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.207931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.208198 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.209128 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.209134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.209361 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.209549 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.209924 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 22:44:10.210678 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.157818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.21.27.188:35985)
0809 22:44:10.161079 (+ 3261us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.161087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.161109 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:10.161209 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.161216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.161368 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.163200 (+ 1832us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.163501 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.165268 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.170474 (+ 5206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.171694 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.179378 (+ 7684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.179774 (+ 396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.179783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.179790 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.179834 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.206207 (+ 26373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.207535 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.207541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.207545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.207777 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.208434 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.208439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.208443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.208937 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.209077 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.209664 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.209901 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3122,"thread_start_us":50724,"threads_started":1}
I20260809 22:44:10.212319 28263 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.27.188:35985: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2d0d1f658c624c139355f60c372624d3"
member_type: NON_VOTER
last_known_addr {
host: "127.21.27.188"
port: 35985
}
attrs {
promote: true
}
}
dest_uuid: "6e0ad208e06c4cf1b71b3252018ee991"
cas_config_opid_index: 5
I20260809 22:44:10.213562 28263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } }
W20260809 22:44:10.215426 28988 tablet.cc:2369] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:10.224328 29069 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2d0d1f658c624c139355f60c372624d3 (127.21.27.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.226276 29010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:10.230378 29010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:44:10.231299 29045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:10.231637 29045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } }
I20260809 22:44:10.233367 29045 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945), cf413bf632fb435e820449fc27926bce (127.21.27.189:41517)
I20260809 22:44:10.234067 28922 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:44:10.234150 28273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0d1f658c624c139355f60c372624d3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0ad208e06c4cf1b71b3252018ee991" is_pre_election: true
I20260809 22:44:10.235020 28921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0d1f658c624c139355f60c372624d3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf413bf632fb435e820449fc27926bce" is_pre_election: true
I20260809 22:44:10.237149 28984 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d0d1f658c624c139355f60c372624d3; no voters: 6e0ad208e06c4cf1b71b3252018ee991, cf413bf632fb435e820449fc27926bce
I20260809 22:44:10.238020 29045 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:44:10.243870 29069 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.27.188:35985
I20260809 22:44:10.244031 29071 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 22:44:10.244580 28867 master.cc:537] Master@127.21.27.188:35985 shutting down...
I20260809 22:44:10.245467 29068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.250265 29068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:10.257707 29070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 22:44:10.268040 28921 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2d0d1f658c624c139355f60c372624d3 (127.21.27.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.273272 29075 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: NON_VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: true } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.273736 29075 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: LEARNER
I20260809 22:44:10.274080 29068 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: attempting to promote NON_VOTER cf413bf632fb435e820449fc27926bce to VOTER
I20260809 22:44:10.275002 29068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } }
I20260809 22:44:10.278175 28867 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:10.278323 28921 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 LEARNER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 22:44:10.278692 28867 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:10.278920 28867 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: stopping tablet replica
I20260809 22:44:10.279100 29068 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 22:44:10.282315 29072 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf413bf632fb435e820449fc27926bce (127.21.27.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.286157 28922 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf413bf632fb435e820449fc27926bce (127.21.27.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
W20260809 22:44:10.291821 28248 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 -> Peer 2d0d1f658c624c139355f60c372624d3 (127.21.27.188:35985): Couldn't send request to peer 2d0d1f658c624c139355f60c372624d3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 22:44:10.295847 29072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.296777 29072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:10.296499 29068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.297099 29068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:10.298454 29075 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.298832 29075 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:10.300313 28867 master.cc:552] Master@127.21.27.188:35985 shutdown complete.
I20260809 22:44:10.300403 28867 master_runner.cc:305] Clearing existing system tablet
I20260809 22:44:10.300820 28867 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:10.300951 28867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:10.304086 28867 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 22:44:10.306066 29078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:10.306502 28867 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:10.306931 28867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:44:10.307093 28867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "2d0d1f658c624c139355f60c372624d3"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "3f7f0ad6634b463a09e5e76bf8e7f034"
server_key_iv: "95ee9511561e199da1b152a6330e7954"
server_key_version: "encryptionkey@0"
I20260809 22:44:10.313141 28867 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:10.323244 28867 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:44:10.323639 28867 log.cc:1196] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 22:44:10.324661 28867 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Deleting consensus metadata
I20260809 22:44:10.325031 28867 master_runner.cc:315] Copying system tablet from 127.21.27.190:39945
I20260809 22:44:10.327677 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28329
I20260809 22:44:10.333438 28867 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: tablet copy: Beginning tablet copy session from remote peer at address 127.21.27.190:39945
I20260809 22:44:10.351650 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.334481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.21.27.190:39945)
0809 22:44:10.334800 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.334807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.335628 (+ 821us) server_negotiation.cc:407] Connection header received
0809 22:44:10.335895 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.335904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.336006 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.336131 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.337421 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.338741 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.340705 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.341597 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.341753 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.344210 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.344219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.344227 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.344278 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.348336 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.349200 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.349208 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.349214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.349383 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.349856 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.349864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.349870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.350300 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.350462 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.350958 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.351270 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260809 22:44:10.351692 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.334406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:43740)
0809 22:44:10.335467 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.335511 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.335707 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.336181 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.336186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.336954 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.337238 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.337253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.338913 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.338919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.340540 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.340556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.341765 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.341771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.341965 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.342746 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.342771 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.343968 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.348493 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.348499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.348504 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.349036 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.349509 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.349514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.349518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.349722 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.350434 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.350440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.350653 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.350805 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.351199 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"thread_start_us":670,"threads_started":1}
I20260809 22:44:10.352242 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:42039
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:39945
--webserver_port=38025
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:44:10.352677 28284 tablet_copy_service.cc:140] P 6e0ad208e06c4cf1b71b3252018ee991: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2d0d1f658c624c139355f60c372624d3 ({username='oryx', principal='oryx/127.21.27.188@KRBTEST.COM'} at 127.0.0.1:43740)
I20260809 22:44:10.352854 28284 tablet_copy_service.cc:161] P 6e0ad208e06c4cf1b71b3252018ee991: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2d0d1f658c624c139355f60c372624d3 at {username='oryx', principal='oryx/127.21.27.188@KRBTEST.COM'} at 127.0.0.1:43740: session id = 2d0d1f658c624c139355f60c372624d3-00000000000000000000000000000000
I20260809 22:44:10.356612 28284 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:44:10.358805 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:44:10.366191 28867 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: tablet copy: Starting download of 0 data blocks...
I20260809 22:44:10.366530 28867 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: tablet copy: Starting download of 1 WAL segments...
I20260809 22:44:10.369484 28867 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:44:10.374248 28284 tablet_copy_service.cc:342] P 6e0ad208e06c4cf1b71b3252018ee991: Request end of tablet copy session 2d0d1f658c624c139355f60c372624d3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.27.188@KRBTEST.COM'} at 127.0.0.1:43740
I20260809 22:44:10.374425 28284 tablet_copy_service.cc:433] P 6e0ad208e06c4cf1b71b3252018ee991: ending tablet copy session 2d0d1f658c624c139355f60c372624d3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2d0d1f658c624c139355f60c372624d3
I20260809 22:44:10.387327 28867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:10.392362 29086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:10.393478 29087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:10.409376 29089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:10.409823 28867 server_base.cc:1034] running on GCE node
I20260809 22:44:10.410211 28867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:10.411012 28867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:10.412209 28867 hybrid_clock.cc:648] HybridClock initialized: now 1786315450412159 us; error 51 us; skew 500 ppm
I20260809 22:44:10.412474 28867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315450, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:10.414842 28867 init.cc:377] Logged in from keytab as oryx/127.21.27.188@KRBTEST.COM (short username oryx)
I20260809 22:44:10.420287 28867 webserver.cc:460] Webserver started at http://127.21.27.188:42639/ using document root <none> and password file <none>
I20260809 22:44:10.420704 28867 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:10.420825 28867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:10.424338 28867 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 22:44:10.426352 29096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:10.426848 28867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 22:44:10.427019 28867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "2d0d1f658c624c139355f60c372624d3"
format_stamp: "Formatted at 2026-08-09 22:44:09 on dist-test-slave-chsk"
server_key: "3f7f0ad6634b463a09e5e76bf8e7f034"
server_key_iv: "95ee9511561e199da1b152a6330e7954"
server_key_version: "encryptionkey@0"
I20260809 22:44:10.427282 28867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:10.440596 28867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:10.444993 28867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:10.476147 28867 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.188:35985
I20260809 22:44:10.476244 29150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.188:35985 every 8 connection(s)
I20260809 22:44:10.477447 28867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
I20260809 22:44:10.482800 29151 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:44:10.487022 29151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Bootstrap starting.
I20260809 22:44:10.506435 29151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:44:10.506965 29151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: Bootstrap complete.
I20260809 22:44:10.508170 29151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } }
I20260809 22:44:10.508534 29151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2d0d1f658c624c139355f60c372624d3, State: Initialized, Role: LEARNER
I20260809 22:44:10.508839 29151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } }
I20260809 22:44:10.515905 29154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.516227 29151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:10.516381 29154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:44:10.528471 29167 catalog_manager.cc:1259] Loaded cluster ID: e6f3d2d99ee04a7088b423adb0057265
I20260809 22:44:10.528704 29167 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: loading cluster ID for follower catalog manager: success
I20260809 22:44:10.532192 29167 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: acquiring CA information for follower catalog manager: success
I20260809 22:44:10.535059 29167 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:44:10.816431 29169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.808745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.0.0.1:42702)
0809 22:44:10.809337 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.809364 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.809530 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.810157 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.810160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.810192 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:10.810372 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.810379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.812413 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.812417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.813661 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.813694 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.815637 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.815642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.815839 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.815866 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.815957 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.816131 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":270,"threads_started":1}
I20260809 22:44:10.816910 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.808876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.21.27.188:35985)
0809 22:44:10.809770 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.809777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.809800 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:44:10.809881 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.809888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.809982 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.810135 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:10.810530 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.812222 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.813850 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.815359 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.815552 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.815809 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.816051 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.816235 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":543,"threads_started":1}
I20260809 22:44:10.817898 29125 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 22:44:10.818491 29154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0ad208e06c4cf1b71b3252018ee991. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:10.818854 29154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: LEARNER
W20260809 22:44:10.826000 29085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:10.826335 29085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:10.826496 29085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:10.834934 29085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:10.835059 29085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:10.835258 29085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:10.845430 29085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:42039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=38025
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:10.846771 29085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:10.848361 29085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:10.866179 29178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:10.866081 29179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:10.866117 29181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:10.867714 29085 server_base.cc:1034] running on GCE node
I20260809 22:44:10.868402 29085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:10.869457 29085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:10.870705 29085 hybrid_clock.cc:648] HybridClock initialized: now 1786315450870657 us; error 43 us; skew 500 ppm
I20260809 22:44:10.871045 29085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315450, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:10.875221 29085 init.cc:377] Logged in from keytab as oryx/127.21.27.129@KRBTEST.COM (short username oryx)
I20260809 22:44:10.877113 29085 webserver.cc:460] Webserver started at http://127.21.27.129:38025/ using document root <none> and password file <none>
I20260809 22:44:10.877806 29085 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:10.877925 29085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:10.883697 29085 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:44:10.886441 29188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:10.887822 29085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:10.888002 29085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "ad94b7c949834fba88ef428449347086"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "7aeb142ce33c337b3fec61c3a08f3086"
server_key_iv: "492a243952562ea8d8293bcfee8e910f"
server_key_version: "encryptionkey@0"
I20260809 22:44:10.889047 29085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:10.908984 29085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:10.913931 29085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:10.914240 29085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:10.915400 29085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:10.916906 29085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:10.916996 29085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:10.917137 29085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:10.917227 29085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:10.956121 29085 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:42039
I20260809 22:44:10.956223 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:42039 every 8 connection(s)
I20260809 22:44:10.957733 29085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:10.966158 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 29085
I20260809 22:44:10.966806 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28472
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315450, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315450, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.130@KRBTEST.COM
Aug 09 22:44:10 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315450, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:10.993109 29170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.961132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:46269 (local address 127.21.27.188:35985)
0809 22:44:10.961344 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.961350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.963201 (+ 1851us) server_negotiation.cc:407] Connection header received
0809 22:44:10.970055 (+ 6854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.970061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.970167 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.970310 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.972575 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.973427 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.978481 (+ 5054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.979287 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.979396 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.984812 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.984820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.984826 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.984875 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.988627 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.989699 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.989705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.989710 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.989905 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.990568 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.990575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.990578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.990990 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.991197 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.992280 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.992550 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:44:10.994962 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.961416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:56469 (local address 127.21.27.189:41517)
0809 22:44:10.962102 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.962110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.974854 (+ 12744us) server_negotiation.cc:407] Connection header received
0809 22:44:10.974960 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.974968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.975085 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.975261 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.977790 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.979005 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.981513 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.982171 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.982288 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.987079 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.987085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.987089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.987173 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.990811 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.991998 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.992004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.992007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.992264 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.992948 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.992957 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.992962 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.993672 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.993830 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.994385 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 22:44:10.994624 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":345,"threads_started":1}
I20260809 22:44:10.995564 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.961449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.21.27.129:46269)
0809 22:44:10.962876 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.962992 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.969685 (+ 6693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.970582 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.970591 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.971424 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.972370 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.972390 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.973639 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.973646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.978214 (+ 4568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.978232 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.979804 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.979812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.980137 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.981042 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.981077 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.984471 (+ 3394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.988799 (+ 4328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.988811 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.988834 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.989520 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.990100 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.990107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.990113 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.990400 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.991137 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.991171 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.991711 (+ 540us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.993290 (+ 1579us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.993980 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":517,"threads_started":1}
I20260809 22:44:10.995611 29311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.961449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.21.27.129:56469)
0809 22:44:10.967959 (+ 6510us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.967989 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.970103 (+ 2114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.975310 (+ 5207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.975324 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.976787 (+ 1463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.977574 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.977591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.979222 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.979234 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.981288 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.981308 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.982545 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.982552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.982739 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.983550 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.983584 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.986834 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.990957 (+ 4123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.990962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.990966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.991823 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.992446 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.992450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.992454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.992626 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.993844 (+ 1218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.993849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.994063 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 22:44:10.994225 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:44:10.994558 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6281,"mutex_wait_us":361,"thread_start_us":5624,"threads_started":1}
I20260809 22:44:10.997593 29304 heartbeater.cc:347] Connected to a master server at 127.21.27.188:35985
I20260809 22:44:10.997671 29306 heartbeater.cc:347] Connected to a master server at 127.21.27.189:41517
I20260809 22:44:10.998011 29304 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:10.998009 29306 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:10.999029 29306 heartbeater.cc:511] Master 127.21.27.189:41517 requested a full tablet report, sending...
I20260809 22:44:10.999594 29304 heartbeater.cc:511] Master 127.21.27.188:35985 requested a full tablet report, sending...
I20260809 22:44:11.000537 29313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.968258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50937 (local address 127.21.27.190:39945)
0809 22:44:10.968987 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.968996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.977673 (+ 8677us) server_negotiation.cc:407] Connection header received
0809 22:44:10.977947 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.977953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.978047 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.978132 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.979498 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.980663 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.982696 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.983504 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.983604 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.992249 (+ 8645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.992257 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.992262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.992308 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.996985 (+ 4677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.997848 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.997854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.997857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.998035 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.998563 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.998568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.998571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.998962 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:10.999109 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:10.999791 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.000045 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":287,"threads_started":1}
I20260809 22:44:11.001570 29315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.969148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:50937)
0809 22:44:10.977548 (+ 8400us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:10.977580 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:10.977785 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:10.978356 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:10.978360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:10.978888 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:10.979301 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.979314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.980852 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.980858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.982535 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:10.982546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.983927 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.983935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:10.984187 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.985727 (+ 1540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:10.985755 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:10.991954 (+ 6199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:10.997131 (+ 5177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.997138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.997141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.997693 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.998211 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:10.998216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:10.998219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:10.998403 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:10.999225 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:10.999248 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:10.999469 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 22:44:11.000834 (+ 1365us) client_negotiation.cc:240] Negotiation successful
0809 22:44:11.001156 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8200,"mutex_wait_us":1,"thread_start_us":7910,"threads_started":1}
I20260809 22:44:11.002483 28912 ts_manager.cc:194] Registered new tserver with Master: ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
I20260809 22:44:11.002599 29113 ts_manager.cc:194] Registered new tserver with Master: ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
I20260809 22:44:11.002770 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:10.968961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.21.27.130:43961)
0809 22:44:10.969761 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:10.969770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:10.976692 (+ 6922us) server_negotiation.cc:407] Connection header received
0809 22:44:10.976861 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:10.976873 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:10.977273 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:10.977445 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:10.980630 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.982253 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.984282 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:10.985041 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:10.985119 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:10.991847 (+ 6728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:10.991869 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:10.991891 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:10.991994 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:10.998224 (+ 6230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.999265 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.999271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.999273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:10.999467 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:10.999975 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:10.999983 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:10.999988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.000452 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:11.000784 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:11.002035 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.002340 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":435,"threads_started":1}
I20260809 22:44:11.003691 29305 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:11.003821 29305 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:11.004281 29305 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:11.004444 28602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:11.005684 28263 ts_manager.cc:194] Re-registered known tserver with Master: ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
I20260809 22:44:11.007392 28263 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:50937
I20260809 22:44:11.091375 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28472
I20260809 22:44:11.110752 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:43961
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:39945
--webserver_port=41825
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:44:11.411293 29174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: NON_VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: true } } }
I20260809 22:44:11.411725 29174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:11.413434 29318 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: attempting to promote NON_VOTER 2d0d1f658c624c139355f60c372624d3 to VOTER
I20260809 22:44:11.414011 29318 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } }
I20260809 22:44:11.416323 28921 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:44:11.416339 29125 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 LEARNER]: Refusing update from remote peer 6e0ad208e06c4cf1b71b3252018ee991: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:44:11.417044 29319 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:44:11.417526 29318 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:44:11.420622 29319 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d0d1f658c624c139355f60c372624d3 (127.21.27.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.421037 28921 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d0d1f658c624c139355f60c372624d3 (127.21.27.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.423115 29125 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d0d1f658c624c139355f60c372624d3 (127.21.27.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.425064 29319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.425340 29319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:11.425284 29174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.425520 29174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:11.425722 29319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.425945 29319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:11.427770 29325 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.428257 29325 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:11.431607 29326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e0ad208e06c4cf1b71b3252018ee991" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:11.432049 29326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 22:44:11.452893 29317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:11.453186 29317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:11.453385 29317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:11.461935 29317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:11.462091 29317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:11.462234 29317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:11.472244 29317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:43961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=41825
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:11.473596 29317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:11.475167 29317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:11.490199 29331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:11.490720 29332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:11.492132 29335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:11.493152 29317 server_base.cc:1034] running on GCE node
I20260809 22:44:11.493883 29317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:11.494989 29317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:11.496197 29317 hybrid_clock.cc:648] HybridClock initialized: now 1786315451496136 us; error 71 us; skew 500 ppm
I20260809 22:44:11.496531 29317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:11 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315451, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:11.499753 29317 init.cc:377] Logged in from keytab as oryx/127.21.27.130@KRBTEST.COM (short username oryx)
I20260809 22:44:11.501397 29317 webserver.cc:460] Webserver started at http://127.21.27.130:41825/ using document root <none> and password file <none>
I20260809 22:44:11.502259 29317 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:11.502427 29317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:11.507982 29317 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:44:11.510962 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:11.512292 29317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:44:11.512465 29317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "58951a3020cd4913a8fb4430901fa508"
format_stamp: "Formatted at 2026-08-09 22:44:07 on dist-test-slave-chsk"
server_key: "34946459c9ce71b0a909b96a0a6ff366"
server_key_iv: "4034348fea8f1689ca74bf9527538719"
server_key_version: "encryptionkey@0"
I20260809 22:44:11.513459 29317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:11.541689 29317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:11.546448 29317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:11.546759 29317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:11.547811 29317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:11.549180 29317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:11.549314 29317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:11.549437 29317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:11.549525 29317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:11.584700 29317 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:43961
I20260809 22:44:11.584790 29461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:43961 every 8 connection(s)
I20260809 22:44:11.586134 29317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:11.596019 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 29317
I20260809 22:44:11.596408 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28613
Aug 09 22:44:11 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.131@KRBTEST.COM
Aug 09 22:44:11 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315451, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
Aug 09 22:44:11 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315451, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
Aug 09 22:44:11 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315451, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
I20260809 22:44:11.640103 29473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.597467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.21.27.131:45727)
0809 22:44:11.612937 (+ 15470us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:11.612951 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:11.614132 (+ 1181us) server_negotiation.cc:407] Connection header received
0809 22:44:11.614951 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:11.614966 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:11.615363 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:11.615567 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:11.617836 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.619017 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.620417 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.621138 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.621279 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.629696 (+ 8417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:11.629717 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:11.629740 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:11.630728 (+ 988us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:11.634889 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.636408 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.636417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.636421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.636599 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.637153 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.637160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.637165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.637889 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:11.638225 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:11.639356 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.639605 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15191,"thread_start_us":15502,"threads_started":1}
I20260809 22:44:11.641634 28744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:11.650360 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.621804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.21.27.130:52379)
0809 22:44:11.628386 (+ 6582us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:11.628404 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:11.628521 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:11.630583 (+ 2062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:11.630590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:11.631286 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:11.631900 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.631922 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.633622 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.633630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.635276 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.635293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.636773 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.636781 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.637086 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.637790 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:11.637816 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:11.641873 (+ 4057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:11.646343 (+ 4470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.646349 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.646355 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.646933 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.647430 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.647435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.647438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.647780 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.648538 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:11.648542 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:11.648850 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 22:44:11.649016 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 22:44:11.649269 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6461,"mutex_wait_us":1,"spinlock_wait_cycles":25344,"thread_start_us":6814,"threads_started":1}
I20260809 22:44:11.650375 29470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.590792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:48787 (local address 127.21.27.188:35985)
0809 22:44:11.617175 (+ 26383us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:11.617195 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:11.620985 (+ 3790us) server_negotiation.cc:407] Connection header received
0809 22:44:11.625318 (+ 4333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:11.625327 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:11.625442 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:11.625563 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:11.629034 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.630533 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.633376 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.634244 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.634336 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.640715 (+ 6379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:11.640722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:11.640726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:11.640767 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:11.644646 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.645682 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.645688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.645691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.645844 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.646405 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.646412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.646418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.646820 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:11.646973 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:11.648630 (+ 1657us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.649917 (+ 1287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26211,"thread_start_us":26411,"threads_started":1}
I20260809 22:44:11.650481 29475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.622323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:52379 (local address 127.21.27.189:41517)
0809 22:44:11.630248 (+ 7925us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:11.630254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:11.630275 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:44:11.630332 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:11.630337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:11.630405 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:11.632082 (+ 1677us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:11.632268 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.633457 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.635471 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.636433 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.637463 (+ 1030us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.642132 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:11.642141 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:11.642146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:11.642190 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:11.646194 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.647084 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.647088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.647091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.647293 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.647944 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.647951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.647956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.648387 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:11.649583 (+ 1196us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:11.649775 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.649997 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7797,"thread_start_us":7632,"threads_started":1}
I20260809 22:44:11.650395 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.590174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.21.27.130:48787)
0809 22:44:11.620867 (+ 30693us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:11.620887 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:11.625110 (+ 4223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:11.626156 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:11.626173 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:11.627620 (+ 1447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:11.628830 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.628859 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.630737 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.630742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.633192 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.633204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.634743 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.634749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.635183 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.636370 (+ 1187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:11.636390 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:11.640477 (+ 4087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:11.644837 (+ 4360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.644849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.644869 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.645513 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.646003 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.646007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.646012 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.646272 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.647050 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:11.647061 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:11.647613 (+ 552us) client_negotiation.cc:769] Sending connection context
0809 22:44:11.648007 (+ 394us) client_negotiation.cc:240] Negotiation successful
0809 22:44:11.648723 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30533,"mutex_wait_us":143,"spinlock_wait_cycles":20736,"thread_start_us":30771,"threads_started":1}
I20260809 22:44:11.652297 29465 heartbeater.cc:347] Connected to a master server at 127.21.27.189:41517
I20260809 22:44:11.652343 29463 heartbeater.cc:347] Connected to a master server at 127.21.27.188:35985
I20260809 22:44:11.652662 29465 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:11.652676 29463 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:11.653568 29465 heartbeater.cc:511] Master 127.21.27.189:41517 requested a full tablet report, sending...
I20260809 22:44:11.653614 29463 heartbeater.cc:511] Master 127.21.27.188:35985 requested a full tablet report, sending...
I20260809 22:44:11.655714 28912 ts_manager.cc:194] Registered new tserver with Master: 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961)
I20260809 22:44:11.656736 29113 ts_manager.cc:194] Registered new tserver with Master: 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961)
I20260809 22:44:11.657526 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.589152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:38599 (local address 127.21.27.190:39945)
0809 22:44:11.616539 (+ 27387us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:11.616545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:11.618338 (+ 1793us) server_negotiation.cc:407] Connection header received
0809 22:44:11.625028 (+ 6690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:11.625038 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:11.625160 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:11.625308 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:11.629862 (+ 4554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.630827 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.633401 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.634154 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.634308 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.640710 (+ 6402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:11.640715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:11.640719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:11.640767 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:11.646391 (+ 5624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.651235 (+ 4844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.651243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.651249 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.651441 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:11.655603 (+ 4162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:11.655613 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:11.655617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:11.655990 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:11.656162 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:11.656887 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 22:44:11.657141 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27249,"thread_start_us":27010,"threads_started":1}
I20260809 22:44:11.657734 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:11.589740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:38599)
0809 22:44:11.618090 (+ 28350us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:11.618159 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:11.624640 (+ 6481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:11.628970 (+ 4330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:11.628977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:11.629444 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:11.629702 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.629714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.631270 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.631280 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.633190 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:11.633204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:11.634670 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:11.634690 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:11.635188 (+ 498us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:11.636092 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:11.636146 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:11.640454 (+ 4308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:11.647014 (+ 6560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.647050 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.647084 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.650741 (+ 3657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.655251 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:11.655261 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:11.655263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:11.655463 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:11.656401 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:11.656406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:11.656590 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 22:44:11.656755 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 22:44:11.657126 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27912,"mutex_wait_us":169,"thread_start_us":30735,"threads_started":1}
I20260809 22:44:11.658722 29464 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:11.658870 29464 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:11.659530 29464 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:11.660706 28263 ts_manager.cc:194] Re-registered known tserver with Master: 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961)
I20260809 22:44:11.662107 28263 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:38599
I20260809 22:44:11.744632 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28613
I20260809 22:44:11.762897 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:45727
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:39945
--webserver_port=45915
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:44:12.010416 29305 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
W20260809 22:44:12.109133 29477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:12.109462 29477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:12.109629 29477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:12.117022 29477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:12.117168 29477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:12.117373 29477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:12.127439 29477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:45727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=45915
--tserver_master_addrs=127.21.27.190:39945,127.21.27.189:41517,127.21.27.188:35985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:12.128964 29477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:12.130334 29477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:12.143746 29483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:12.143666 29484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:12.144160 29486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:12.144644 29477 server_base.cc:1034] running on GCE node
I20260809 22:44:12.145294 29477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:12.146242 29477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:12.147461 29477 hybrid_clock.cc:648] HybridClock initialized: now 1786315452147421 us; error 35 us; skew 500 ppm
I20260809 22:44:12.147742 29477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:12 dist-test-slave-chsk krb5kdc[28214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315452, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:12.150859 29477 init.cc:377] Logged in from keytab as oryx/127.21.27.131@KRBTEST.COM (short username oryx)
I20260809 22:44:12.152333 29477 webserver.cc:460] Webserver started at http://127.21.27.131:45915/ using document root <none> and password file <none>
I20260809 22:44:12.152999 29477 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:12.153162 29477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:12.158861 29477 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:44:12.161689 29495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:12.163129 29477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:44:12.163333 29477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
format_stamp: "Formatted at 2026-08-09 22:44:08 on dist-test-slave-chsk"
server_key: "8469676da8d1e153e35b3316d357c245"
server_key_iv: "6892369d99e56f760d3e88a9e5453c1c"
server_key_version: "encryptionkey@0"
I20260809 22:44:12.164250 29477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:12.190311 29477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:12.194641 29477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:12.194964 29477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:12.195955 29477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:12.197407 29477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:12.197528 29477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:12.197657 29477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:12.197752 29477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:12.230821 29477 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45727
I20260809 22:44:12.230908 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45727 every 8 connection(s)
I20260809 22:44:12.232280 29477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:12.238351 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 29477
Aug 09 22:44:12 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315452, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.189@KRBTEST.COM
Aug 09 22:44:12 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315452, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
Aug 09 22:44:12 dist-test-slave-chsk krb5kdc[28214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315452, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.188@KRBTEST.COM
I20260809 22:44:12.305716 29628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.261816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.21.27.190:39945)
0809 22:44:12.273817 (+ 12001us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.273825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.273847 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:12.273923 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.273931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.274031 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.274200 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.285606 (+ 11406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.288058 (+ 2452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.290640 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.292244 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.292384 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.296712 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.296719 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.296724 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.296763 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.301239 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.303243 (+ 2004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.303252 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.303258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.303464 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.303878 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.303884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.303888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.304330 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.304602 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.305035 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.305281 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":180,"server-negotiator.queue_time_us":11833,"thread_start_us":12568,"threads_started":1}
I20260809 22:44:12.309495 29634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.268128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.21.27.188:35985)
0809 22:44:12.270317 (+ 2189us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.270323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.271870 (+ 1547us) server_negotiation.cc:407] Connection header received
0809 22:44:12.271964 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.271973 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.272098 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.272223 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.278571 (+ 6348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.280358 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.283047 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.284056 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.284211 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.298174 (+ 13963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.298183 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.298188 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.298247 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.305376 (+ 7129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.306232 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.306238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.306241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.306400 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.306874 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.306880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.306883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.307359 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.307579 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.308796 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.309081 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2029,"thread_start_us":1899,"threads_started":1}
I20260809 22:44:12.315442 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.260678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:35283 (local address 127.21.27.190:39945)
0809 22:44:12.261175 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.261182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.272454 (+ 11272us) server_negotiation.cc:407] Connection header received
0809 22:44:12.279509 (+ 7055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.279514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.279614 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.279736 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.286344 (+ 6608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.287484 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.290979 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.291636 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.291771 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.304204 (+ 12433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.304211 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.304217 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.304273 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.308272 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.309831 (+ 1559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.309839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.309843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.311277 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.312792 (+ 1515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.312797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.312800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.313192 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.313395 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.314719 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.314974 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5579,"server-negotiator.queue_time_us":353,"thread_start_us":242,"threads_started":1}
I20260809 22:44:12.315992 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.260549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:35283)
0809 22:44:12.268030 (+ 7481us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.268049 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.279325 (+ 11276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.284059 (+ 4734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.284067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.284823 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:12.286190 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.286206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.288902 (+ 2696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.288909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.290855 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.290869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.293643 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.293656 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.293945 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.295380 (+ 1435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:12.295413 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:12.303914 (+ 8501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:12.308429 (+ 4515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.308439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.308457 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.309214 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.312373 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.312379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.312384 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.312673 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.313376 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:12.313386 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:12.313818 (+ 432us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.314106 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.314648 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7349,"thread_start_us":7216,"threads_started":1}
I20260809 22:44:12.319698 29620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.235814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.21.27.131:58355)
0809 22:44:12.261052 (+ 25238us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.261072 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.274918 (+ 13846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.292108 (+ 17190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.292113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.292755 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:12.293107 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.294274 (+ 1167us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.295568 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.295575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.297119 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.297133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.298516 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.298523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.298746 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.299719 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:12.299747 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:12.305122 (+ 5375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:12.316498 (+ 11376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.316503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.316508 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.317015 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.317479 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.317484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.317488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.317690 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.318451 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:12.318456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:12.318631 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.318771 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.319075 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23444,"thread_start_us":24082,"threads_started":1}
I20260809 22:44:12.321065 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.235023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:58355 (local address 127.21.27.189:41517)
0809 22:44:12.285756 (+ 50733us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.285761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.285780 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:44:12.285844 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.285852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.285953 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.286095 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.294471 (+ 8376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.295371 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.297322 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.298121 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.298264 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.306300 (+ 8036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.306308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.306312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.306358 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.316332 (+ 9974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.317165 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.317169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.317172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.317345 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.317824 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.317829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.317832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.318226 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.318376 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.320069 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.320349 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50605,"thread_start_us":50466,"threads_started":1}
I20260809 22:44:12.321873 29614 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39945
I20260809 22:44:12.322237 29615 heartbeater.cc:347] Connected to a master server at 127.21.27.189:41517
I20260809 22:44:12.322392 29615 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:12.322268 29614 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:12.323674 29615 heartbeater.cc:511] Master 127.21.27.189:41517 requested a full tablet report, sending...
I20260809 22:44:12.323764 29614 heartbeater.cc:511] Master 127.21.27.190:39945 requested a full tablet report, sending...
I20260809 22:44:12.313325 28263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 22:44:12.326257 28262 ts_manager.cc:194] Re-registered known tserver with Master: c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
W20260809 22:44:12.327978 28263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:12.328408 28262 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:35283
I20260809 22:44:12.329648 28912 ts_manager.cc:194] Registered new tserver with Master: c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
I20260809 22:44:12.333228 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.236491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.21.27.131:58479)
0809 22:44:12.260213 (+ 23722us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.260266 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.276299 (+ 16033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.279040 (+ 2741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.279059 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.280573 (+ 1514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:12.281671 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.281702 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.285694 (+ 3992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.285700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.288682 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.288702 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.299114 (+ 10412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.299122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.299365 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.301493 (+ 2128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:12.301515 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:12.313520 (+ 12005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:12.327430 (+ 13910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.327436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.327439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.328074 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.328578 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:12.328584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:12.328588 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:12.328745 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:12.332082 (+ 3337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:12.332088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:12.332260 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.332408 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.332763 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23507,"thread_start_us":22448,"threads_started":1}
I20260809 22:44:12.334337 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.235110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:58479 (local address 127.21.27.188:35985)
0809 22:44:12.247537 (+ 12427us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.247547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.268214 (+ 20667us) server_negotiation.cc:407] Connection header received
0809 22:44:12.278053 (+ 9839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.278063 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.278201 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.278322 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.281953 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.283811 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.291052 (+ 7241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.291781 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.291896 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.313854 (+ 21958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.313861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.313866 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.313905 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.326885 (+ 12980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.328249 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.328255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.328258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.328429 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.328888 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.328895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.328900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.331832 (+ 2932us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.331997 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.333557 (+ 1560us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.333807 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12240,"thread_start_us":12398,"threads_started":1}
I20260809 22:44:12.335479 29613 heartbeater.cc:347] Connected to a master server at 127.21.27.188:35985
I20260809 22:44:12.335613 29613 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:12.336071 29613 heartbeater.cc:511] Master 127.21.27.188:35985 requested a full tablet report, sending...
I20260809 22:44:12.337790 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.285722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.21.27.189:41517)
0809 22:44:12.302735 (+ 17013us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.302742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.302760 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:12.302808 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.302813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.302898 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.307421 (+ 4523us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.310877 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.312192 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.317857 (+ 5665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.318629 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.318742 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.325523 (+ 6781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.325532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.325537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.325614 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.330968 (+ 5354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.332581 (+ 1613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.332586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.332590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.332755 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.335116 (+ 2361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.335122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.335126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.335572 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.335810 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.337065 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.337339 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16859,"thread_start_us":16730,"threads_started":1}
I20260809 22:44:12.340517 29113 ts_manager.cc:194] Registered new tserver with Master: c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
I20260809 22:44:12.375313 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.352431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:42039 (local address 127.0.0.1:38302)
0809 22:44:12.363942 (+ 11511us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.363961 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.366203 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.367456 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.367463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.367489 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:12.367865 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.367881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.370167 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.370177 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.372650 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.372664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.374325 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.374330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.374478 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.374500 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.374632 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.374787 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11352,"thread_start_us":11119,"threads_started":1}
I20260809 22:44:12.375285 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.353048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.21.27.129:42039)
0809 22:44:12.365534 (+ 12486us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.365542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.365569 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:44:12.366438 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.366453 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.366807 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.367182 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:12.368105 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.369954 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.372833 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.373897 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.373978 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.374138 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.374707 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.374935 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12309,"thread_start_us":12189,"threads_started":1}
I20260809 22:44:12.379773 29237 tablet_service.cc:1467] Processing CreateTablet for tablet 627abe64fa0c449494f33988968d8428 (DEFAULT_TABLE table=test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:12.380971 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.352619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:43961 (local address 127.0.0.1:49122)
0809 22:44:12.369089 (+ 16470us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.369110 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.369226 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.372620 (+ 3394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.372625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.372650 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:12.373356 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.373369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.376115 (+ 2746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.376120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.378262 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.378277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.380020 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.380026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.380227 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.380262 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.380363 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.380505 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14744,"mutex_wait_us":30,"thread_start_us":14596,"threads_started":1}
I20260809 22:44:12.381376 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.351650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45727 (local address 127.0.0.1:59500)
0809 22:44:12.358506 (+ 6856us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.358534 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.358726 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.374062 (+ 15336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.374066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.374092 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:12.374498 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.374511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.377086 (+ 2575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.377091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.378813 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.378825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.380483 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.380491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.380701 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.380732 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.380839 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.380987 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":27,"thread_start_us":386,"threads_started":1}
I20260809 22:44:12.381398 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.352113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.21.27.131:45727)
0809 22:44:12.373593 (+ 21480us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.373600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.373630 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:44:12.373721 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.373727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.373883 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.374072 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:12.375263 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.376646 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.378968 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.380269 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.380394 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.380571 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.380911 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.381097 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21291,"thread_start_us":21388,"threads_started":1}
I20260809 22:44:12.381455 29237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627abe64fa0c449494f33988968d8428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:12.382084 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.352904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.21.27.130:43961)
0809 22:44:12.371834 (+ 18930us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.371845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.371879 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:44:12.372008 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.372019 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.372396 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.372629 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:12.374386 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.375998 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.378455 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.379801 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.379954 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.380143 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.381617 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.381786 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18727,"thread_start_us":18517,"threads_started":1}
I20260809 22:44:12.384902 29547 tablet_service.cc:1467] Processing CreateTablet for tablet 627abe64fa0c449494f33988968d8428 (DEFAULT_TABLE table=test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:12.385318 29392 tablet_service.cc:1467] Processing CreateTablet for tablet 627abe64fa0c449494f33988968d8428 (DEFAULT_TABLE table=test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:12.386086 29547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627abe64fa0c449494f33988968d8428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:12.386572 29392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627abe64fa0c449494f33988968d8428. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:12.397655 29653 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Bootstrap starting.
I20260809 22:44:12.401279 29655 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Bootstrap starting.
I20260809 22:44:12.403375 29653 tablet_bootstrap.cc:654] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:12.403708 29654 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Bootstrap starting.
I20260809 22:44:12.404086 29655 tablet_bootstrap.cc:654] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:12.404718 29653 log.cc:824] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:12.405066 29655 log.cc:824] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:12.407130 29654 tablet_bootstrap.cc:654] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:12.408509 29653 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: No bootstrap required, opened a new log
I20260809 22:44:12.408489 29654 log.cc:824] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:12.408924 29653 ts_tablet_manager.cc:1397] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260809 22:44:12.409463 29655 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: No bootstrap required, opened a new log
I20260809 22:44:12.409822 29655 ts_tablet_manager.cc:1397] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 22:44:12.412968 29654 tablet_bootstrap.cc:492] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: No bootstrap required, opened a new log
I20260809 22:44:12.413312 29654 ts_tablet_manager.cc:1397] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260809 22:44:12.419081 29655 raft_consensus.cc:348] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.419642 29655 raft_consensus.cc:374] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:12.419800 29655 raft_consensus.cc:729] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58951a3020cd4913a8fb4430901fa508, State: Initialized, Role: FOLLOWER
I20260809 22:44:12.420588 29655 consensus_queue.cc:260] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.423076 29655 ts_tablet_manager.cc:1428] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260809 22:44:12.423440 29464 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
I20260809 22:44:12.423641 29653 raft_consensus.cc:348] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.424232 29653 raft_consensus.cc:374] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:12.424410 29653 raft_consensus.cc:729] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad94b7c949834fba88ef428449347086, State: Initialized, Role: FOLLOWER
I20260809 22:44:12.425259 29653 consensus_queue.cc:260] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.432454 29654 raft_consensus.cc:348] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.432999 29654 raft_consensus.cc:374] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:12.433069 29653 ts_tablet_manager.cc:1428] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Time spent starting tablet: real 0.024s user 0.016s sys 0.008s
I20260809 22:44:12.433244 29654 raft_consensus.cc:729] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a03013fcd44d95b841f3ad9ee0f106, State: Initialized, Role: FOLLOWER
I20260809 22:44:12.434258 29654 consensus_queue.cc:260] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.436882 29614 heartbeater.cc:503] Master 127.21.27.190:39945 was elected leader, sending a full tablet report...
I20260809 22:44:12.439098 29654 ts_tablet_manager.cc:1428] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Time spent starting tablet: real 0.026s user 0.014s sys 0.004s
I20260809 22:44:12.459993 29659 raft_consensus.cc:482] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:12.460466 29659 raft_consensus.cc:504] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.463040 29659 leader_election.cc:288] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727), ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
W20260809 22:44:12.470463 29307 tablet.cc:2369] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:12.475589 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.463372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:55675 (local address 127.21.27.131:45727)
0809 22:44:12.463581 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.463585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.465148 (+ 1563us) server_negotiation.cc:407] Connection header received
0809 22:44:12.465676 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.465683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.465777 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.465861 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:12.467180 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.468760 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.472680 (+ 3920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.474302 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.474584 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.474694 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.474969 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.475094 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:44:12.476002 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.463701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45727 (local address 127.21.27.130:55675)
0809 22:44:12.464972 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.465024 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.465384 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.466152 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.466157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.466196 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:12.466963 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.466985 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.468975 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.468981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.472438 (+ 3457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.472465 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.474609 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.474616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.474801 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.474826 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.474941 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.475076 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":66,"thread_start_us":695,"threads_started":1}
I20260809 22:44:12.476770 29652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.465400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43013 (local address 127.21.27.129:42039)
0809 22:44:12.465591 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.465596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.465944 (+ 348us) server_negotiation.cc:407] Connection header received
0809 22:44:12.466162 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.466165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.466224 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.466301 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:12.467491 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.468907 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.473178 (+ 4271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.474896 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.475040 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.475116 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.476212 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.476371 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 22:44:12.477654 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.464815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:42039 (local address 127.21.27.130:43013)
0809 22:44:12.465845 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:12.465861 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:12.466011 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:12.466366 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:12.466375 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:12.466408 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:12.467331 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.467341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.469042 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.469085 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.472982 (+ 3897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:12.472994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.475494 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.475502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:12.475877 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.475911 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:44:12.476947 (+ 1036us) client_negotiation.cc:240] Negotiation successful
0809 22:44:12.477080 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":116,"thread_start_us":793,"threads_started":1}
I20260809 22:44:12.479317 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627abe64fa0c449494f33988968d8428" candidate_uuid: "58951a3020cd4913a8fb4430901fa508" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" is_pre_election: true
I20260809 22:44:12.479933 29258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627abe64fa0c449494f33988968d8428" candidate_uuid: "58951a3020cd4913a8fb4430901fa508" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad94b7c949834fba88ef428449347086" is_pre_election: true
I20260809 22:44:12.480129 29567 raft_consensus.cc:2454] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58951a3020cd4913a8fb4430901fa508 in term 0.
I20260809 22:44:12.480558 29258 raft_consensus.cc:2454] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58951a3020cd4913a8fb4430901fa508 in term 0.
I20260809 22:44:12.481151 29347 leader_election.cc:302] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58951a3020cd4913a8fb4430901fa508, c7a03013fcd44d95b841f3ad9ee0f106; no voters:
I20260809 22:44:12.481734 29659 raft_consensus.cc:2781] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:12.481931 29659 raft_consensus.cc:482] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:12.482092 29659 raft_consensus.cc:3037] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:12.485227 29659 raft_consensus.cc:504] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.486075 29659 leader_election.cc:288] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [CANDIDATE]: Term 1 election: Requested vote from peers c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727), ad94b7c949834fba88ef428449347086 (127.21.27.129:42039)
I20260809 22:44:12.486644 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627abe64fa0c449494f33988968d8428" candidate_uuid: "58951a3020cd4913a8fb4430901fa508" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
I20260809 22:44:12.486917 29567 raft_consensus.cc:3037] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:12.487387 29258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627abe64fa0c449494f33988968d8428" candidate_uuid: "58951a3020cd4913a8fb4430901fa508" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad94b7c949834fba88ef428449347086"
I20260809 22:44:12.487645 29258 raft_consensus.cc:3037] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:12.490072 29567 raft_consensus.cc:2454] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58951a3020cd4913a8fb4430901fa508 in term 1.
I20260809 22:44:12.490525 29347 leader_election.cc:302] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58951a3020cd4913a8fb4430901fa508, c7a03013fcd44d95b841f3ad9ee0f106; no voters:
I20260809 22:44:12.490965 29659 raft_consensus.cc:2781] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:12.491930 29659 raft_consensus.cc:686] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 1 LEADER]: Becoming Leader. State: Replica: 58951a3020cd4913a8fb4430901fa508, State: Running, Role: LEADER
I20260809 22:44:12.492254 29258 raft_consensus.cc:2454] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58951a3020cd4913a8fb4430901fa508 in term 1.
I20260809 22:44:12.492596 29659 consensus_queue.cc:237] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } }
I20260809 22:44:12.499907 28263 catalog_manager.cc:5495] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58951a3020cd4913a8fb4430901fa508 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "58951a3020cd4913a8fb4430901fa508" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } health_report { overall_health: HEALTHY } } }
W20260809 22:44:12.502595 29616 tablet.cc:2369] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:12.535995 29650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.528278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.21.27.130:43961)
0809 22:44:12.528535 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.528543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.528862 (+ 319us) server_negotiation.cc:407] Connection header received
0809 22:44:12.529063 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.529067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.529147 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.529278 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:12.529881 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.530937 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.533297 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.533908 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.534009 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.534746 (+ 737us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:12.535309 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:12.535455 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.535545 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.535610 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:44:12.545457 29567 raft_consensus.cc:1261] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Refusing update from remote peer 58951a3020cd4913a8fb4430901fa508: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:12.545789 29258 raft_consensus.cc:1261] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Refusing update from remote peer 58951a3020cd4913a8fb4430901fa508: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:12.546435 29666 consensus_queue.cc:1035] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:12.546893 29659 consensus_queue.cc:1035] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:12.590751 28262 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:12.591188 28262 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:12.600298 28262 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e0ad208e06c4cf1b71b3252018ee991: Sending DeleteTablet for 3 replicas of tablet 627abe64fa0c449494f33988968d8428
I20260809 22:44:12.602658 29392 tablet_service.cc:1514] Processing DeleteTablet for tablet 627abe64fa0c449494f33988968d8428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:12 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:49122
I20260809 22:44:12.602855 29547 tablet_service.cc:1514] Processing DeleteTablet for tablet 627abe64fa0c449494f33988968d8428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:12 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:59500
I20260809 22:44:12.603001 29237 tablet_service.cc:1514] Processing DeleteTablet for tablet 627abe64fa0c449494f33988968d8428 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:12 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:38302
I20260809 22:44:12.604907 29680 tablet_replica.cc:330] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: stopping tablet replica
I20260809 22:44:12.605263 29681 tablet_replica.cc:330] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: stopping tablet replica
I20260809 22:44:12.605728 29681 raft_consensus.cc:2229] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:12.606362 29681 raft_consensus.cc:2258] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:12.606616 29682 tablet_replica.cc:330] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: stopping tablet replica
I20260809 22:44:12.606448 29680 raft_consensus.cc:2229] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:12.607105 29682 raft_consensus.cc:2229] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:12.608006 29680 raft_consensus.cc:2258] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:12.608501 29681 ts_tablet_manager.cc:1905] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:12.608901 29682 raft_consensus.cc:2258] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:12.611624 29680 ts_tablet_manager.cc:1905] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:12.614193 29682 ts_tablet_manager.cc:1905] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:12.615185 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28228
I20260809 22:44:12.616443 28295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:12.624969 29682 ts_tablet_manager.cc:1918] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:12.625412 29682 log.cc:1196] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/627abe64fa0c449494f33988968d8428
I20260809 22:44:12.626530 29682 ts_tablet_manager.cc:1939] T 627abe64fa0c449494f33988968d8428 P ad94b7c949834fba88ef428449347086: Deleting consensus metadata
I20260809 22:44:12.629869 29681 ts_tablet_manager.cc:1918] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:12.630252 29681 log.cc:1196] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/627abe64fa0c449494f33988968d8428
I20260809 22:44:12.631217 29681 ts_tablet_manager.cc:1939] T 627abe64fa0c449494f33988968d8428 P 58951a3020cd4913a8fb4430901fa508: Deleting consensus metadata
I20260809 22:44:12.631772 29680 ts_tablet_manager.cc:1918] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:12.632141 29680 log.cc:1196] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/627abe64fa0c449494f33988968d8428
I20260809 22:44:12.633201 29680 ts_tablet_manager.cc:1939] T 627abe64fa0c449494f33988968d8428 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting consensus metadata
I20260809 22:44:12.646908 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.605160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.21.27.189:41517)
0809 22:44:12.605400 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.605407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.612785 (+ 7378us) server_negotiation.cc:407] Connection header received
0809 22:44:12.613004 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.613009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.613110 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.613217 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.627523 (+ 14306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.628964 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.632014 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.633287 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.633433 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.638512 (+ 5079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.638518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.638521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.638567 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.643076 (+ 4509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.644135 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.644141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.644145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.644376 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.644881 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.644889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.644892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.645271 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.645618 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.646111 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.646452 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:44:12.667011 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.613195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.21.27.188:35985)
0809 22:44:12.613365 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.613372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.647094 (+ 33722us) server_negotiation.cc:407] Connection header received
0809 22:44:12.647415 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.647421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.647583 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.647651 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.650628 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.651595 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.653396 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.654069 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.654226 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.658242 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.658248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.658251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.658288 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.662901 (+ 4613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.664318 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.664324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.664328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.664523 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.664963 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.664970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.664974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.665389 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.665802 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.666138 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.666488 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 22:44:12.789044 28249 catalog_manager.cc:4841] TS ad94b7c949834fba88ef428449347086 (127.21.27.129:42039): tablet 627abe64fa0c449494f33988968d8428 (table test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]) successfully deleted
W20260809 22:44:12.789451 28249 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.27.129:42039, user_credentials={real_user=slave}} blocked reactor thread for 160303us
I20260809 22:44:12.789757 28246 catalog_manager.cc:4841] TS 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961): tablet 627abe64fa0c449494f33988968d8428 (table test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]) successfully deleted
W20260809 22:44:12.789973 28246 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.27.130:43961, user_credentials={real_user=slave}} blocked reactor thread for 156367us
I20260809 22:44:12.790194 28249 catalog_manager.cc:4841] TS c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727): tablet 627abe64fa0c449494f33988968d8428 (table test-table [id=b0519029a5c045cc8c3ff095ffff8bf5]) successfully deleted
I20260809 22:44:12.790761 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28228
I20260809 22:44:12.842607 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.822877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.21.27.188:35985)
0809 22:44:12.823222 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.823232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.824170 (+ 938us) server_negotiation.cc:407] Connection header received
0809 22:44:12.824431 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.824437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.824524 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.824618 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.827059 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.828245 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.830620 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.831312 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.831441 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.835080 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.835087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.835092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.835176 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.839215 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.840117 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.840123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.840126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.840291 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.840888 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.840893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.840896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.841293 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.841565 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.841969 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.842219 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260809 22:44:12.842693 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:12.824373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.21.27.189:41517)
0809 22:44:12.824561 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:12.824567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:12.824824 (+ 257us) server_negotiation.cc:407] Connection header received
0809 22:44:12.824976 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:12.824982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:12.825065 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:12.825171 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:12.827104 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.828139 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.831645 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:12.832399 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:12.832529 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:12.835718 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:12.835723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:12.835726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:12.835772 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:12.838884 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.839920 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.839926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.839930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.840093 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:12.840476 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:12.840482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:12.840485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:12.840784 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:12.841074 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:12.842123 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0809 22:44:12.842351 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:13.573808 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:13.572972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:38161)
0809 22:44:13.573400 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:13.573545 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":227,"threads_started":1}
W20260809 22:44:13.574280 29464 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:39945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:13.576426 29694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:13.575693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:41295)
0809 22:44:13.576098 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:13.576217 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":224,"threads_started":1}
W20260809 22:44:13.576841 29305 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:39945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:13.580219 29695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:13.579280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:60619)
0809 22:44:13.579771 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:13.579919 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":266,"threads_started":1}
W20260809 22:44:13.581066 29614 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:39945 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.100066 29696 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e0ad208e06c4cf1b71b3252018ee991)
I20260809 22:44:14.100322 29696 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } }
I20260809 22:44:14.101186 29696 leader_election.cc:288] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945), 2d0d1f658c624c139355f60c372624d3 (127.21.27.188:35985)
I20260809 22:44:14.103910 29698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6e0ad208e06c4cf1b71b3252018ee991)
I20260809 22:44:14.104240 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.101351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44178)
0809 22:44:14.103684 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.103904 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2152,"thread_start_us":1990,"threads_started":1}
I20260809 22:44:14.104164 29698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } }
W20260809 22:44:14.104596 28895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.105264 29698 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945), cf413bf632fb435e820449fc27926bce (127.21.27.189:41517)
I20260809 22:44:14.107048 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.106631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44186)
0809 22:44:14.106786 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.106889 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 22:44:14.107257 28895 leader_election.cc:334] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945): Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.108286 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.106602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44182)
0809 22:44:14.107764 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.107984 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1022,"thread_start_us":315,"threads_started":1}
W20260809 22:44:14.109192 29098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.111454 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.110863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44192)
0809 22:44:14.111044 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.111248 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:44:14.111699 29098 leader_election.cc:334] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945): Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.113677 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.103896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42730 (local address 127.21.27.188:35985)
0809 22:44:14.106580 (+ 2684us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.106585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.106602 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:44:14.106657 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.106684 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.106763 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.106880 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.107743 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.109109 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.111567 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.112652 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.112798 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.112901 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.113181 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.113319 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2547,"thread_start_us":2416,"threads_started":1}
I20260809 22:44:14.113771 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.101721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:35985 (local address 127.0.0.1:42730)
0809 22:44:14.105926 (+ 4205us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.105952 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.106156 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.107060 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.107065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.107100 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.107536 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.107566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.109282 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.109288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.111420 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.111433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.112817 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.112823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.113011 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.113035 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.113134 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.113269 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4058,"mutex_wait_us":5,"thread_start_us":3831,"threads_started":1}
I20260809 22:44:14.114369 29125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf413bf632fb435e820449fc27926bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d0d1f658c624c139355f60c372624d3" is_pre_election: true
I20260809 22:44:14.114778 29125 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf413bf632fb435e820449fc27926bce in term 1.
I20260809 22:44:14.115336 28896 leader_election.cc:302] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d0d1f658c624c139355f60c372624d3, cf413bf632fb435e820449fc27926bce; no voters: 6e0ad208e06c4cf1b71b3252018ee991
I20260809 22:44:14.115477 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.107896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:41517 (local address 127.0.0.1:59654)
0809 22:44:14.108488 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.108515 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.108622 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.109781 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.109786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.109818 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.110101 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.110118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.111780 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.111785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.113187 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.113200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.114778 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.114782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.114922 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.114975 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.115087 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.115269 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":932,"threads_started":1}
I20260809 22:44:14.115756 29696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:44:14.115816 29705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.107863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.21.27.189:41517)
0809 22:44:14.109431 (+ 1568us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.109439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.109465 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:44:14.109532 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.109535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.109585 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.109754 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.110319 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.111611 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.113336 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.114555 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.114689 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.114806 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.115115 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.115277 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":1428,"thread_start_us":1220,"threads_started":1}
I20260809 22:44:14.115940 29696 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6e0ad208e06c4cf1b71b3252018ee991)
I20260809 22:44:14.116071 29696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:44:14.116400 28921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d0d1f658c624c139355f60c372624d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cf413bf632fb435e820449fc27926bce" is_pre_election: true
I20260809 22:44:14.118511 29696 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } }
I20260809 22:44:14.118960 28921 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2d0d1f658c624c139355f60c372624d3 in current term 2: Already voted for candidate cf413bf632fb435e820449fc27926bce in this term.
I20260809 22:44:14.119220 29696 leader_election.cc:288] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 election: Requested vote from peers 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945), 2d0d1f658c624c139355f60c372624d3 (127.21.27.188:35985)
I20260809 22:44:14.119784 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.119295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44206)
0809 22:44:14.119515 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.119614 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 22:44:14.119882 29125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf413bf632fb435e820449fc27926bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d0d1f658c624c139355f60c372624d3"
I20260809 22:44:14.120020 29098 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d0d1f658c624c139355f60c372624d3; no voters: 6e0ad208e06c4cf1b71b3252018ee991, cf413bf632fb435e820449fc27926bce
I20260809 22:44:14.120133 29125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:44:14.120937 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.120634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44216)
0809 22:44:14.120766 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.120830 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:44:14.121112 28895 leader_election.cc:334] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945): Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
I20260809 22:44:14.122701 29125 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf413bf632fb435e820449fc27926bce in term 2.
I20260809 22:44:14.122937 29698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:44:14.123183 28896 leader_election.cc:302] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d0d1f658c624c139355f60c372624d3, cf413bf632fb435e820449fc27926bce; no voters: 6e0ad208e06c4cf1b71b3252018ee991
I20260809 22:44:14.123476 29696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:44:14.123929 29696 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [term 2 LEADER]: Becoming Leader. State: Replica: cf413bf632fb435e820449fc27926bce, State: Running, Role: LEADER
I20260809 22:44:14.124307 29696 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } }
I20260809 22:44:14.126128 29707 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf413bf632fb435e820449fc27926bce. Latest consensus state: current_term: 2 leader_uuid: "cf413bf632fb435e820449fc27926bce" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:14.126469 29707 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:14.127121 29710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:14.131379 29710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:14.132215 29710 catalog_manager.cc:1259] Loaded cluster ID: e6f3d2d99ee04a7088b423adb0057265
I20260809 22:44:14.132324 29710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:14.133463 29710 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:14.134295 29710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Loaded TSK: 0
I20260809 22:44:14.135106 29710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:44:14.189541 28912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:14.196695 28912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:14.200656 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.200304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44250)
0809 22:44:14.200444 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.200522 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:44:14.200835 29125 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [term 2 FOLLOWER]: Refusing update from remote peer cf413bf632fb435e820449fc27926bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 22:44:14.201689 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.201268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44266)
0809 22:44:14.201473 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.201537 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:44:14.201586 29707 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 22:44:14.201890 28895 consensus_peers.cc:487] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce -> Peer 6e0ad208e06c4cf1b71b3252018ee991 (127.21.27.190:39945): Couldn't send request to peer 6e0ad208e06c4cf1b71b3252018ee991. Status: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:44:14.204246 29698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf413bf632fb435e820449fc27926bce. Latest consensus state: current_term: 2 leader_uuid: "cf413bf632fb435e820449fc27926bce" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:14.204489 29698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:14.206092 29696 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cf413bf632fb435e820449fc27926bce" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:14.206442 29696 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:14.206835 29698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cf413bf632fb435e820449fc27926bce" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6e0ad208e06c4cf1b71b3252018ee991" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39945 } } peers { permanent_uuid: "cf413bf632fb435e820449fc27926bce" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 41517 } attrs { promote: false } } peers { permanent_uuid: "2d0d1f658c624c139355f60c372624d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 35985 } attrs { promote: false } } }
I20260809 22:44:14.207077 29698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0d1f658c624c139355f60c372624d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:44:14.207832 29712 mvcc.cc:205] Tried to move back new op lower bound from 7316748100399562752 to 7316748100096819200. Current Snapshot: MvccSnapshot[applied={T|T < 7316748100399562752 or (T in {7316748100399562752})}]
I20260809 22:44:14.225701 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.214925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:42039 (local address 127.0.0.1:38312)
0809 22:44:14.215250 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.215271 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.215375 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.216806 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.216814 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.216851 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.217151 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.217166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.219376 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.219385 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.222131 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.222152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.224620 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.224630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.224925 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.224985 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.225091 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.225287 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 22:44:14.228147 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.214963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.21.27.129:42039)
0809 22:44:14.216431 (+ 1468us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.216437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.216459 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:14.216533 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.216538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.216610 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.217377 (+ 767us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.217662 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.219175 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.222602 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.224360 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.227280 (+ 2920us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.227458 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.227522 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.227734 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216,"thread_start_us":1112,"threads_started":1}
I20260809 22:44:14.229038 29237 tablet_service.cc:1467] Processing CreateTablet for tablet 770700f27d454cc490e861f933ca4ae3 (DEFAULT_TABLE table=test-table [id=d905a252919d474b8f15c58d43d1bc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:14.229581 29237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770700f27d454cc490e861f933ca4ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:14.235872 29699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.215946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45727 (local address 127.0.0.1:59510)
0809 22:44:14.225930 (+ 9984us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.225947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.226047 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.229055 (+ 3008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.229063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.229102 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.229373 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.229387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.231332 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.231337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.233215 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.233234 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.234917 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.234923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.235235 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.235297 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.235377 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.235510 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9900,"mutex_wait_us":26,"thread_start_us":10503,"threads_started":1}
I20260809 22:44:14.236874 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.216536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59510 (local address 127.21.27.131:45727)
0809 22:44:14.228689 (+ 12153us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.228695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.228717 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:14.228790 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.228795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.228881 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.228986 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.229576 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.231184 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.233439 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.234715 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.236061 (+ 1346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.236190 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.236258 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.236456 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12019,"thread_start_us":11927,"threads_started":1}
I20260809 22:44:14.237635 29723 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Bootstrap starting.
I20260809 22:44:14.237797 29547 tablet_service.cc:1467] Processing CreateTablet for tablet 770700f27d454cc490e861f933ca4ae3 (DEFAULT_TABLE table=test-table [id=d905a252919d474b8f15c58d43d1bc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:14.238011 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.215403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:43961 (local address 127.0.0.1:49136)
0809 22:44:14.215584 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.215606 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.215735 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.230470 (+ 14735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.230475 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.230502 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.230782 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.230793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.233116 (+ 2323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.233122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.235170 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.235184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.237265 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.237274 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.237465 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.237497 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.237595 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.237741 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":3}
I20260809 22:44:14.238241 29547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770700f27d454cc490e861f933ca4ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:14.238716 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.215574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.21.27.130:43961)
0809 22:44:14.230150 (+ 14576us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.230155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.230177 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:14.230249 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.230254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.230335 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.230444 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.231941 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.232960 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.235724 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.236878 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.237040 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.237151 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.238227 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.238401 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14432,"thread_start_us":14972,"threads_started":1}
I20260809 22:44:14.239439 29392 tablet_service.cc:1467] Processing CreateTablet for tablet 770700f27d454cc490e861f933ca4ae3 (DEFAULT_TABLE table=test-table [id=d905a252919d474b8f15c58d43d1bc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:14.239847 29723 tablet_bootstrap.cc:654] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:14.239884 29392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 770700f27d454cc490e861f933ca4ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:14.243606 29725 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Bootstrap starting.
I20260809 22:44:14.244987 29725 tablet_bootstrap.cc:654] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:14.247767 29725 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: No bootstrap required, opened a new log
I20260809 22:44:14.247931 29725 ts_tablet_manager.cc:1397] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 22:44:14.248375 29727 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Bootstrap starting.
I20260809 22:44:14.248471 29723 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: No bootstrap required, opened a new log
I20260809 22:44:14.248638 29723 ts_tablet_manager.cc:1397] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 22:44:14.248792 29725 raft_consensus.cc:348] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.249027 29725 raft_consensus.cc:374] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:14.249132 29725 raft_consensus.cc:729] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a03013fcd44d95b841f3ad9ee0f106, State: Initialized, Role: FOLLOWER
I20260809 22:44:14.249421 29723 raft_consensus.cc:348] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.249439 29725 consensus_queue.cc:260] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.249625 29723 raft_consensus.cc:374] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:14.249732 29723 raft_consensus.cc:729] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad94b7c949834fba88ef428449347086, State: Initialized, Role: FOLLOWER
I20260809 22:44:14.250029 29723 consensus_queue.cc:260] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.250348 29725 ts_tablet_manager.cc:1428] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 22:44:14.250389 29727 tablet_bootstrap.cc:654] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:14.258291 29723 ts_tablet_manager.cc:1428] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Time spent starting tablet: real 0.010s user 0.003s sys 0.004s
I20260809 22:44:14.259511 29729 raft_consensus.cc:482] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:14.259778 29729 raft_consensus.cc:504] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.262409 29727 tablet_bootstrap.cc:492] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: No bootstrap required, opened a new log
I20260809 22:44:14.262632 29727 ts_tablet_manager.cc:1397] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260809 22:44:14.263887 29727 raft_consensus.cc:348] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.264228 29727 raft_consensus.cc:374] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:14.264349 29727 raft_consensus.cc:729] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58951a3020cd4913a8fb4430901fa508, State: Initialized, Role: FOLLOWER
I20260809 22:44:14.264621 29727 consensus_queue.cc:260] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.271442 29727 ts_tablet_manager.cc:1428] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260809 22:44:14.273185 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.272234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:44859)
0809 22:44:14.272766 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.272905 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"threads_started":1}
I20260809 22:44:14.275642 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.275225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:58393)
0809 22:44:14.275446 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.275528 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 22:44:14.276098 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.259355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:60087)
0809 22:44:14.275633 (+ 16278us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.275819 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16152,"threads_started":1}
I20260809 22:44:14.276842 29465 heartbeater.cc:503] Master 127.21.27.189:41517 was elected leader, sending a full tablet report...
I20260809 22:44:14.278475 29306 heartbeater.cc:503] Master 127.21.27.189:41517 was elected leader, sending a full tablet report...
I20260809 22:44:14.279510 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.251075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:59021)
0809 22:44:14.279036 (+ 27961us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.279232 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27769,"threads_started":1}
I20260809 22:44:14.281436 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.281094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:49361)
0809 22:44:14.281218 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.281288 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 22:44:14.281589 29615 heartbeater.cc:503] Master 127.21.27.189:41517 was elected leader, sending a full tablet report...
I20260809 22:44:14.285156 29729 leader_election.cc:288] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961), c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
I20260809 22:44:14.287533 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.265318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:38077 (local address 127.21.27.130:43961)
0809 22:44:14.265510 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.265519 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.276411 (+ 10892us) server_negotiation.cc:407] Connection header received
0809 22:44:14.277626 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.277631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.277712 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.277790 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.278566 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.279704 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.284483 (+ 4779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.286028 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.286239 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.286356 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.287056 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.287233 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 22:44:14.287614 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.264822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:43961 (local address 127.21.27.129:38077)
0809 22:44:14.276293 (+ 11471us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.276319 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.276464 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.278013 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.278020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.278055 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.278429 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.278442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.281422 (+ 2980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.281432 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.284258 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.284279 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.286557 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.286566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.286837 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.286891 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.286996 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.287222 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11385,"mutex_wait_us":3,"threads_started":1}
I20260809 22:44:14.288136 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.277407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:59871)
0809 22:44:14.287817 (+ 10410us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.287935 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10287}
I20260809 22:44:14.289583 29415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770700f27d454cc490e861f933ca4ae3" candidate_uuid: "ad94b7c949834fba88ef428449347086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58951a3020cd4913a8fb4430901fa508" is_pre_election: true
I20260809 22:44:14.290042 29415 raft_consensus.cc:2454] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad94b7c949834fba88ef428449347086 in term 0.
I20260809 22:44:14.290727 29190 leader_election.cc:302] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58951a3020cd4913a8fb4430901fa508, ad94b7c949834fba88ef428449347086; no voters:
I20260809 22:44:14.291158 29729 raft_consensus.cc:2781] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:14.291312 29729 raft_consensus.cc:482] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:14.291417 29729 raft_consensus.cc:3037] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:14.294518 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.280280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50925 (local address 127.21.27.131:45727)
0809 22:44:14.280493 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.280500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.287908 (+ 7408us) server_negotiation.cc:407] Connection header received
0809 22:44:14.288133 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.288140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.288224 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.288313 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:14.289172 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.290086 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.292368 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.293592 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.293749 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.293818 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.294131 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.294282 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:44:14.294703 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.271294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45727 (local address 127.21.27.129:50925)
0809 22:44:14.287803 (+ 16509us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.287823 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:14.287994 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:14.288569 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:14.288573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:14.288601 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:14.289002 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.289016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.290259 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.290266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.292221 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:14.292233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.293815 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.293819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:14.293946 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.293969 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:44:14.294110 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:44:14.294222 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16243,"thread_start_us":16070,"threads_started":1}
I20260809 22:44:14.294781 29729 raft_consensus.cc:504] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.295534 29729 leader_election.cc:288] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [CANDIDATE]: Term 1 election: Requested vote from peers 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961), c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727)
I20260809 22:44:14.295502 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770700f27d454cc490e861f933ca4ae3" candidate_uuid: "ad94b7c949834fba88ef428449347086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" is_pre_election: true
I20260809 22:44:14.295750 29567 raft_consensus.cc:2454] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad94b7c949834fba88ef428449347086 in term 0.
I20260809 22:44:14.296070 29566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770700f27d454cc490e861f933ca4ae3" candidate_uuid: "ad94b7c949834fba88ef428449347086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7a03013fcd44d95b841f3ad9ee0f106"
I20260809 22:44:14.296238 29415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "770700f27d454cc490e861f933ca4ae3" candidate_uuid: "ad94b7c949834fba88ef428449347086" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58951a3020cd4913a8fb4430901fa508"
I20260809 22:44:14.296334 29566 raft_consensus.cc:3037] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:14.296449 29415 raft_consensus.cc:3037] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:14.299449 29566 raft_consensus.cc:2454] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad94b7c949834fba88ef428449347086 in term 1.
I20260809 22:44:14.299469 29415 raft_consensus.cc:2454] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad94b7c949834fba88ef428449347086 in term 1.
I20260809 22:44:14.299894 29190 leader_election.cc:302] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58951a3020cd4913a8fb4430901fa508, ad94b7c949834fba88ef428449347086; no voters:
I20260809 22:44:14.300172 29729 raft_consensus.cc:2781] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:14.300359 29729 raft_consensus.cc:686] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 1 LEADER]: Becoming Leader. State: Replica: ad94b7c949834fba88ef428449347086, State: Running, Role: LEADER
I20260809 22:44:14.300954 29729 consensus_queue.cc:237] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } }
I20260809 22:44:14.301308 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.300829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:60721)
0809 22:44:14.300981 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.301141 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 22:44:14.304638 28911 catalog_manager.cc:5495] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad94b7c949834fba88ef428449347086 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "ad94b7c949834fba88ef428449347086" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad94b7c949834fba88ef428449347086" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 } health_report { overall_health: UNKNOWN } } }
I20260809 22:44:14.336257 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.331324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.21.27.129:42039)
0809 22:44:14.331532 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.331537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.331555 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:14.331728 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.331732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.331819 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.331942 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:14.332254 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.332941 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.334194 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.334589 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.334681 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.335068 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:14.335658 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:14.335768 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.335881 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.335944 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:44:14.341526 29566 raft_consensus.cc:1261] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Refusing update from remote peer ad94b7c949834fba88ef428449347086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:14.341828 29415 raft_consensus.cc:1261] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Refusing update from remote peer ad94b7c949834fba88ef428449347086: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:14.342337 29732 consensus_queue.cc:1035] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7a03013fcd44d95b841f3ad9ee0f106" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:14.342782 29729 consensus_queue.cc:1035] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58951a3020cd4913a8fb4430901fa508" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 43961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:14.346091 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.345583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:39387)
0809 22:44:14.345735 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.345824 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 22:44:14.346710 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.346061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:44481)
0809 22:44:14.346395 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.346497 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 22:44:14.354597 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.352688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:45115)
0809 22:44:14.354269 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.354359 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1439}
I20260809 22:44:14.354619 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.353894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.130:56623)
0809 22:44:14.354268 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.354389 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260809 22:44:14.360128 29744 mvcc.cc:205] Tried to move back new op lower bound from 7316748100974665728 to 7316748100820856832. Current Snapshot: MvccSnapshot[applied={T|T < 7316748100974665728}]
I20260809 22:44:14.369287 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.368700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.129:45001)
0809 22:44:14.368955 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.369072 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 22:44:14.372061 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.371517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.21.27.131:52309)
0809 22:44:14.371745 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.371857 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 22:44:14.374980 28912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:14.375263 28912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:14.381147 28912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf413bf632fb435e820449fc27926bce: Sending DeleteTablet for 3 replicas of tablet 770700f27d454cc490e861f933ca4ae3
I20260809 22:44:14.382081 29237 tablet_service.cc:1514] Processing DeleteTablet for tablet 770700f27d454cc490e861f933ca4ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:14 UTC) from {username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:38312
I20260809 22:44:14.382328 29392 tablet_service.cc:1514] Processing DeleteTablet for tablet 770700f27d454cc490e861f933ca4ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:14 UTC) from {username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:49136
I20260809 22:44:14.383036 29547 tablet_service.cc:1514] Processing DeleteTablet for tablet 770700f27d454cc490e861f933ca4ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:14 UTC) from {username='oryx', principal='oryx/127.21.27.189@KRBTEST.COM'} at 127.0.0.1:59510
I20260809 22:44:14.384517 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 29085
I20260809 22:44:14.397889 29752 tablet_replica.cc:330] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: stopping tablet replica
I20260809 22:44:14.398361 29752 raft_consensus.cc:2229] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:14.399331 29752 raft_consensus.cc:2258] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:14.400219 29753 tablet_replica.cc:330] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: stopping tablet replica
I20260809 22:44:14.400570 29753 raft_consensus.cc:2229] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:14.400797 29752 ts_tablet_manager.cc:1905] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:14.401090 29753 raft_consensus.cc:2258] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:14.401722 29754 tablet_replica.cc:330] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: stopping tablet replica
I20260809 22:44:14.401918 29754 raft_consensus.cc:2229] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:14.402235 29754 raft_consensus.cc:2258] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:14.402503 29753 ts_tablet_manager.cc:1905] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:14.404894 29754 ts_tablet_manager.cc:1905] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:14.407753 29753 ts_tablet_manager.cc:1918] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:14.407920 29753 log.cc:1196] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/770700f27d454cc490e861f933ca4ae3
I20260809 22:44:14.408432 29753 ts_tablet_manager.cc:1939] T 770700f27d454cc490e861f933ca4ae3 P 58951a3020cd4913a8fb4430901fa508: Deleting consensus metadata
I20260809 22:44:14.409777 28894 catalog_manager.cc:4841] TS 58951a3020cd4913a8fb4430901fa508 (127.21.27.130:43961): tablet 770700f27d454cc490e861f933ca4ae3 (table test-table [id=d905a252919d474b8f15c58d43d1bc02]) successfully deleted
I20260809 22:44:14.411767 29754 ts_tablet_manager.cc:1918] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:14.411927 29754 log.cc:1196] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/770700f27d454cc490e861f933ca4ae3
I20260809 22:44:14.412449 29754 ts_tablet_manager.cc:1939] T 770700f27d454cc490e861f933ca4ae3 P c7a03013fcd44d95b841f3ad9ee0f106: Deleting consensus metadata
I20260809 22:44:14.412617 29752 ts_tablet_manager.cc:1918] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:14.412801 29752 log.cc:1196] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/770700f27d454cc490e861f933ca4ae3
I20260809 22:44:14.413332 29752 ts_tablet_manager.cc:1939] T 770700f27d454cc490e861f933ca4ae3 P ad94b7c949834fba88ef428449347086: Deleting consensus metadata
I20260809 22:44:14.413705 28897 catalog_manager.cc:4841] TS c7a03013fcd44d95b841f3ad9ee0f106 (127.21.27.131:45727): tablet 770700f27d454cc490e861f933ca4ae3 (table test-table [id=d905a252919d474b8f15c58d43d1bc02]) successfully deleted
I20260809 22:44:14.414429 28897 catalog_manager.cc:4841] TS ad94b7c949834fba88ef428449347086 (127.21.27.129:42039): tablet 770700f27d454cc490e861f933ca4ae3 (table test-table [id=d905a252919d474b8f15c58d43d1bc02]) successfully deleted
I20260809 22:44:14.418871 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.385721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.21.27.129:42039)
0809 22:44:14.385892 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.385899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.391607 (+ 5708us) server_negotiation.cc:407] Connection header received
0809 22:44:14.391828 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.391832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.391910 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.392023 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:14.401288 (+ 9265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.402348 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.404170 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.404783 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.404912 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.411097 (+ 6185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:14.411117 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:14.411154 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:14.411200 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:14.415645 (+ 4445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.416414 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.416423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.416427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.416641 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.417093 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.417098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.417102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.417553 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:14.417896 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.418235 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.418456 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:44:14.419615 29299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:14.524011 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 29085
I20260809 22:44:14.545946 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 29317
I20260809 22:44:14.559959 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.546776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.21.27.130:43961)
0809 22:44:14.546970 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.546976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.547039 (+ 63us) server_negotiation.cc:407] Connection header received
0809 22:44:14.547287 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.547292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.547358 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.547470 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:14.548833 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.549555 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.550869 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.551335 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.551474 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.554112 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:14.554128 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:14.554144 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:14.554194 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:14.557078 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.557831 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.557838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.557841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.557963 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.558349 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.558354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.558357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.558714 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:14.559058 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.559409 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.559625 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:44:14.560786 29457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:14.656266 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.655718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44278)
0809 22:44:14.655941 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.656078 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:44:14.657596 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.657104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39945 (local address 127.0.0.1:44292)
0809 22:44:14.657293 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:14.657384 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:39945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:44:14.683928 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 29317
I20260809 22:44:14.710121 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 29477
I20260809 22:44:14.724720 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:14.711048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.21.27.131:45727)
0809 22:44:14.711281 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:14.711287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:14.711309 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:14.711412 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:14.711418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:14.711491 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:14.711603 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:14.712989 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.713736 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.714983 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:14.715482 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:14.715621 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:14.718369 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:14.718386 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:14.718402 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:14.718435 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:14.721629 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.722369 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.722376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.722379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.722509 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:14.722847 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:14.722850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:14.722852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:14.723387 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:14.723725 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:14.724061 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 22:44:14.724296 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:44:14.725591 29608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:14.822374 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 29477
I20260809 22:44:14.842535 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28755
I20260809 22:44:14.843663 28944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:14.969652 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28755
W20260809 22:44:14.993706 29098 connection.cc:535] server connection from 127.0.0.1:42730 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:44:14.994170 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 28867
I20260809 22:44:14.995324 29146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:15.113530 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 28867
2026-08-09T22:44:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10052 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:15 dist-test-slave-chsk krb5kdc[29765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:15 dist-test-slave-chsk krb5kdc[29765](info): set up 2 sockets
Aug 09 22:44:15 dist-test-slave-chsk krb5kdc[29765](info): commencing operation
krb5kdc: starting...
W20260809 22:44:16.276644 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:16 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab.
Entry for principal oryx/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab.
I20260809 22:44:16.522523 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37563
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab
--principal=oryx/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:37563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:16.854466 29779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:16.854741 29779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:16.854923 29779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:16.862059 29779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:16.862157 29779 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:44:16.862243 29779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:16.862329 29779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:16.862430 29779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:16.862547 29779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:16.871979 29779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:37563
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.190.keytab
--principal=oryx/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:16.873238 29779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:16.874630 29779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:16.886879 29785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:16.886803 29788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:16.886904 29786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:16.887844 29779 server_base.cc:1034] running on GCE node
I20260809 22:44:16.888638 29779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:16.890134 29779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:16.891427 29779 hybrid_clock.cc:648] HybridClock initialized: now 1786315456891339 us; error 36 us; skew 500 ppm
I20260809 22:44:16.891896 29779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:16 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315456, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:16.895129 29779 init.cc:377] Logged in from keytab as oryx/127.21.27.190@KRBTEST.COM (short username oryx)
I20260809 22:44:16.896663 29779 webserver.cc:460] Webserver started at http://127.21.27.190:39261/ using document root <none> and password file <none>
I20260809 22:44:16.897353 29779 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:16.897442 29779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:16.897727 29779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:16.900785 29779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "370cd7341aed47dcab6f12ac352b7c1e"
format_stamp: "Formatted at 2026-08-09 22:44:16 on dist-test-slave-chsk"
server_key: "573e9368b029af52b4eb295fb9bfa107"
server_key_iv: "3407a5b2dd8622d3c6798298d6ff4590"
server_key_version: "encryptionkey@0"
I20260809 22:44:16.901556 29779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "370cd7341aed47dcab6f12ac352b7c1e"
format_stamp: "Formatted at 2026-08-09 22:44:16 on dist-test-slave-chsk"
server_key: "573e9368b029af52b4eb295fb9bfa107"
server_key_iv: "3407a5b2dd8622d3c6798298d6ff4590"
server_key_version: "encryptionkey@0"
I20260809 22:44:16.906786 29779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:44:16.910238 29797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:16.911553 29779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 22:44:16.911755 29779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "370cd7341aed47dcab6f12ac352b7c1e"
format_stamp: "Formatted at 2026-08-09 22:44:16 on dist-test-slave-chsk"
server_key: "573e9368b029af52b4eb295fb9bfa107"
server_key_iv: "3407a5b2dd8622d3c6798298d6ff4590"
server_key_version: "encryptionkey@0"
I20260809 22:44:16.911990 29779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:16.953296 29779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:16.957521 29779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:16.957834 29779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:16.980803 29779 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:37563
I20260809 22:44:16.980827 29865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:37563 every 8 connection(s)
I20260809 22:44:16.982270 29779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:16.986109 29866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:16.987643 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 29779
I20260809 22:44:16.987876 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:16.988533 21614 external_mini_cluster.cc:1428] Setting key 7d14b9429a0385789ec1037593958b2d
I20260809 22:44:16.997463 29866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Bootstrap starting.
I20260809 22:44:17.000730 29866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:17.001765 29866 log.cc:824] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:44:16 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:17.004954 29866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: No bootstrap required, opened a new log
I20260809 22:44:17.011480 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:16.990616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.21.27.190:37563)
0809 22:44:16.991491 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:16.991503 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:16.991546 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:44:16.993320 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:16.993341 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:16.993704 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:16.994075 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:16.995700 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:16.996965 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:16.998304 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:16.999009 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:16.999246 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:17.003694 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:17.003703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:17.003718 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:17.003758 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:17.007016 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.007820 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.007830 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.007836 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.008013 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.008452 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.008456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.008458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.009017 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:17.009378 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:17.009898 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 22:44:17.010342 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":387,"threads_started":1}
I20260809 22:44:17.013650 29866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } }
I20260809 22:44:17.014065 29866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:17.014197 29866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 370cd7341aed47dcab6f12ac352b7c1e, State: Initialized, Role: FOLLOWER
I20260809 22:44:17.014816 29866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } }
I20260809 22:44:17.015035 29866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:17.015131 29866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:17.015328 29866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:17.017759 29866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } }
I20260809 22:44:17.018287 29866 leader_election.cc:302] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 370cd7341aed47dcab6f12ac352b7c1e; no voters:
I20260809 22:44:17.019049 29866 leader_election.cc:288] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:17.019241 29871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:17.020089 29871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [term 1 LEADER]: Becoming Leader. State: Replica: 370cd7341aed47dcab6f12ac352b7c1e, State: Running, Role: LEADER
I20260809 22:44:17.020684 29871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } }
I20260809 22:44:17.021242 29866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:17.023943 29873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 370cd7341aed47dcab6f12ac352b7c1e. Latest consensus state: current_term: 1 leader_uuid: "370cd7341aed47dcab6f12ac352b7c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } } }
I20260809 22:44:17.024200 29872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "370cd7341aed47dcab6f12ac352b7c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "370cd7341aed47dcab6f12ac352b7c1e" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37563 } } }
I20260809 22:44:17.024484 29873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:17.024533 29872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:17.026407 29880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:17.031591 29880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:17.041742 29880 catalog_manager.cc:1347] Generated new cluster ID: feed7f2d8ddb4f2e8f0b631bc4312414
I20260809 22:44:17.041927 29880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:17.057729 29880 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:17.060029 29880 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:17.073678 29880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Generated new TSK 0
I20260809 22:44:17.074990 29880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
I20260809 22:44:17.138372 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:37563
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:17.480826 29893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:17.481102 29893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:17.481290 29893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:17.488730 29893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:17.488852 29893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:17.488993 29893 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:44:17.489115 29893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:17.499013 29893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:37563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:17.500321 29893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:17.501681 29893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:17.514734 29899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:17.514815 29902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:17.514776 29900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:17.516153 29893 server_base.cc:1034] running on GCE node
I20260809 22:44:17.516831 29893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:17.517817 29893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:17.519075 29893 hybrid_clock.cc:648] HybridClock initialized: now 1786315457518998 us; error 28 us; skew 500 ppm
I20260809 22:44:17.519490 29893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:17 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315457, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:17.522967 29893 init.cc:377] Logged in from keytab as oryx/127.21.27.129@KRBTEST.COM (short username oryx)
I20260809 22:44:17.524631 29893 webserver.cc:460] Webserver started at http://127.21.27.129:37613/ using document root <none> and password file <none>
I20260809 22:44:17.525311 29893 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:17.525403 29893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:17.525751 29893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:17.528898 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "0d001893dbd84c908be23f54d4e7e581"
format_stamp: "Formatted at 2026-08-09 22:44:17 on dist-test-slave-chsk"
server_key: "63812d54aaa4824a9c1f03571b582303"
server_key_iv: "b261ab6057d75dee6a38a1b957270ee4"
server_key_version: "encryptionkey@0"
I20260809 22:44:17.529700 29893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "0d001893dbd84c908be23f54d4e7e581"
format_stamp: "Formatted at 2026-08-09 22:44:17 on dist-test-slave-chsk"
server_key: "63812d54aaa4824a9c1f03571b582303"
server_key_iv: "b261ab6057d75dee6a38a1b957270ee4"
server_key_version: "encryptionkey@0"
I20260809 22:44:17.534945 29893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:44:17.538442 29909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:17.539897 29893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:17.540036 29893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "0d001893dbd84c908be23f54d4e7e581"
format_stamp: "Formatted at 2026-08-09 22:44:17 on dist-test-slave-chsk"
server_key: "63812d54aaa4824a9c1f03571b582303"
server_key_iv: "b261ab6057d75dee6a38a1b957270ee4"
server_key_version: "encryptionkey@0"
I20260809 22:44:17.540228 29893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:17.563683 29893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:17.567730 29893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:17.567970 29893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:17.570724 29893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:17.570848 29893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:17.570984 29893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:17.571066 29893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:44:17 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315457, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:17.594756 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:17.571488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:57633 (local address 127.21.27.190:37563)
0809 22:44:17.572086 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:17.572090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:17.573262 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 22:44:17.575525 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:17.575531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:17.575634 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:17.575758 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:17.577759 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.578876 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.581553 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.582154 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.582305 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:17.587451 (+ 5146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:17.587457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:17.587460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:17.587527 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:17.590988 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.591993 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.591999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.592002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.592136 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.592733 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.592738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.592741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.593145 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:17.593305 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:17.594129 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 22:44:17.594376 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":386,"threads_started":1}
I20260809 22:44:17.596005 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:17.572202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.129:57633)
0809 22:44:17.573063 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:17.573132 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:17.575250 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:17.576006 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:17.576013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:17.576786 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:17.577535 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:17.577564 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.579058 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.579068 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:17.581395 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:17.581407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.582423 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.582431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:17.582657 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:17.583611 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:17.583647 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:17.587169 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:17.591204 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:17.591213 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:17.591234 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:17.591835 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:17.592251 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:17.592258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:17.592263 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:17.592605 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:17.593366 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:17.593372 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:17.593736 (+ 364us) client_negotiation.cc:769] Sending connection context
0809 22:44:17.594028 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 22:44:17.594579 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":372,"threads_started":1}
I20260809 22:44:17.619510 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:17.602060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36633 (local address 127.21.27.190:37563)
0809 22:44:17.602300 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:17.602310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:17.603299 (+ 989us) server_negotiation.cc:407] Connection header received
0809 22:44:17.603507 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:17.603511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:17.603572 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:17.603686 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:17.605301 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.606574 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.609329 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.609908 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.610053 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:17.612869 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:17.612874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:17.612877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:17.612909 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:17.615890 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.616950 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.616956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.616961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.617128 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:17.617705 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:17.617712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:17.617717 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:17.618137 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:17.618337 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:17.618878 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 22:44:17.619105 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:44:17.619585 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:17.602043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.129:36633)
0809 22:44:17.603170 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:17.603196 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:17.603379 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:17.603728 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:17.603735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:17.604564 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:17.605113 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:17.605134 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.606787 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.606795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:17.609139 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:17.609156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:17.610173 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:17.610183 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:17.610445 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:17.611215 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:17.611247 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:17.612658 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:17.616031 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:17.616038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:17.616043 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:17.616776 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:17.617292 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:17.617296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:17.617299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:17.617531 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:17.618277 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:17.618288 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:17.618573 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 22:44:17.618763 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 22:44:17.619118 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":740,"threads_started":1}
I20260809 22:44:17.628566 29893 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:39783
I20260809 22:44:17.628585 30038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:39783 every 8 connection(s)
I20260809 22:44:17.630074 29893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:17.632388 30039 heartbeater.cc:347] Connected to a master server at 127.21.27.190:37563
I20260809 22:44:17.632576 30039 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:17.633131 30039 heartbeater.cc:511] Master 127.21.27.190:37563 requested a full tablet report, sending...
I20260809 22:44:17.634997 29816 ts_manager.cc:194] Registered new tserver with Master: 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783)
I20260809 22:44:17.635257 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 29893
I20260809 22:44:17.635458 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:17.636116 21614 external_mini_cluster.cc:1428] Setting key 49ab077e808ea860b635297d31720929
I20260809 22:44:17.636651 29816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:57633
WARNING: no policy specified for oryx/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
I20260809 22:44:17.689246 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:37563
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:18.019222 30043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:18.019531 30043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:18.019632 30043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:18.027179 30043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:18.027289 30043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:18.027374 30043 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:44:18.027457 30043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:18.036983 30043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:37563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:18.038136 30043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:18.039482 30043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:18.052459 30048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:18.052510 30052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:18.052528 30050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:18.053339 30043 server_base.cc:1034] running on GCE node
I20260809 22:44:18.053881 30043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:18.054708 30043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:18.055918 30043 hybrid_clock.cc:648] HybridClock initialized: now 1786315458055862 us; error 26 us; skew 500 ppm
I20260809 22:44:18.056237 30043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:18.059270 30043 init.cc:377] Logged in from keytab as oryx/127.21.27.130@KRBTEST.COM (short username oryx)
I20260809 22:44:18.060654 30043 webserver.cc:460] Webserver started at http://127.21.27.130:33741/ using document root <none> and password file <none>
I20260809 22:44:18.061357 30043 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:18.061501 30043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:18.061784 30043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:18.064630 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "360667bc032e486590d876fd7e97afbd"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "5b67057c8d070ae2fd07c785392c34b5"
server_key_iv: "6ebf3933781ff9aa94086799ed683a31"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.065382 30043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "360667bc032e486590d876fd7e97afbd"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "5b67057c8d070ae2fd07c785392c34b5"
server_key_iv: "6ebf3933781ff9aa94086799ed683a31"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.070305 30043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:44:18.073555 30059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:18.074813 30043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:18.074942 30043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "360667bc032e486590d876fd7e97afbd"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "5b67057c8d070ae2fd07c785392c34b5"
server_key_iv: "6ebf3933781ff9aa94086799ed683a31"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.075112 30043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:18.106213 30043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:18.110119 30043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:18.110351 30043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:18.112908 30043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:18.112989 30043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:18.113129 30043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:18.113217 30043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:18.138813 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.113858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:47765 (local address 127.21.27.190:37563)
0809 22:44:18.114032 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:18.114036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:18.115599 (+ 1563us) server_negotiation.cc:407] Connection header received
0809 22:44:18.118188 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:18.118196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:18.118282 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:18.118386 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:18.120853 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.121900 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.124788 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.125542 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.125679 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.130726 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:18.130738 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:18.130741 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:18.130775 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:18.134569 (+ 3794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.135597 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.135603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.135606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.135799 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.136520 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.136526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.136532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.136867 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:18.137069 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:18.138226 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0809 22:44:18.138449 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 22:44:18.140422 30068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.114497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.130:47765)
0809 22:44:18.115437 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:18.115473 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:18.117826 (+ 2353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:18.119023 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:18.119031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:18.119801 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:18.120667 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.120692 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.122096 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.122100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.124630 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.124642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.125810 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.125822 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.126233 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.127060 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:18.127100 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:18.130481 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:18.134747 (+ 4266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.134760 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.134777 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.135463 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.135942 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.135949 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.135959 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.136353 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.137167 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:18.137181 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:18.137741 (+ 560us) client_negotiation.cc:769] Sending connection context
0809 22:44:18.138116 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 22:44:18.138667 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":425,"threads_started":1}
I20260809 22:44:18.162709 30043 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:40473
I20260809 22:44:18.162729 30184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:40473 every 8 connection(s)
I20260809 22:44:18.164072 30043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:18.166360 30185 heartbeater.cc:347] Connected to a master server at 127.21.27.190:37563
I20260809 22:44:18.166632 30185 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:18.167397 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.150892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.130:55635)
0809 22:44:18.151699 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:18.151723 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:18.151908 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:18.152329 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:18.152334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:18.152962 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:18.153465 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.153482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.155218 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.155230 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.157533 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.157553 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.158346 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.158356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.158607 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.159475 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:18.159493 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:18.160697 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:18.164005 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.164011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.164015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.164512 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.164915 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.164920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.164924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.165059 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.165702 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:18.165710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:18.165969 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 22:44:18.166158 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 22:44:18.166628 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":407,"threads_started":1}
I20260809 22:44:18.167531 30185 heartbeater.cc:511] Master 127.21.27.190:37563 requested a full tablet report, sending...
I20260809 22:44:18.168314 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.150869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:55635 (local address 127.21.27.190:37563)
0809 22:44:18.151097 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:18.151105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:18.151831 (+ 726us) server_negotiation.cc:407] Connection header received
0809 22:44:18.152060 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:18.152067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:18.152168 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:18.152263 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:18.153697 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.154961 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.157712 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.158122 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.158235 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.160896 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:18.160901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:18.160904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:18.160934 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:18.163871 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.164648 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.164653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.164658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.164799 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.165189 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.165193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.165195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.165546 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:18.165700 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:18.167354 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0809 22:44:18.167986 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:44:18.169116 29816 ts_manager.cc:194] Registered new tserver with Master: 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473)
I20260809 22:44:18.169924 29816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:47765
I20260809 22:44:18.173269 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30043
I20260809 22:44:18.173446 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:18.173904 21614 external_mini_cluster.cc:1428] Setting key 714d2f56a72d20c8d72dedaf13061e9f
WARNING: no policy specified for oryx/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
I20260809 22:44:18.226694 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:37563
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:18.552659 30189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:18.552922 30189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:18.553025 30189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:18.560451 30189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:18.560565 30189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:18.560714 30189 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:44:18.560801 30189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:18.570482 30189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.27.190:37563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:18.571715 30189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:18.572947 30189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:18.586320 30195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:18.586357 30194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:18.586369 30197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:18.587271 30189 server_base.cc:1034] running on GCE node
I20260809 22:44:18.587939 30189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:18.588896 30189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:18.590158 30189 hybrid_clock.cc:648] HybridClock initialized: now 1786315458590096 us; error 36 us; skew 500 ppm
I20260809 22:44:18.590514 30189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:18.593747 30189 init.cc:377] Logged in from keytab as oryx/127.21.27.131@KRBTEST.COM (short username oryx)
I20260809 22:44:18.595293 30189 webserver.cc:460] Webserver started at http://127.21.27.131:36735/ using document root <none> and password file <none>
I20260809 22:44:18.595940 30189 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:18.596028 30189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:18.596311 30189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:18.599409 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "3cb387e7c7f63c905e27337596c85de8"
server_key_iv: "ef33fd4e1d532c382ca0a518d3d9d97c"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.600150 30189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "3cb387e7c7f63c905e27337596c85de8"
server_key_iv: "ef33fd4e1d532c382ca0a518d3d9d97c"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.605099 30189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:44:18.608389 30205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:18.609699 30189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:44:18.609838 30189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
format_stamp: "Formatted at 2026-08-09 22:44:18 on dist-test-slave-chsk"
server_key: "3cb387e7c7f63c905e27337596c85de8"
server_key_iv: "ef33fd4e1d532c382ca0a518d3d9d97c"
server_key_version: "encryptionkey@0"
I20260809 22:44:18.610028 30189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:18.632614 30189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:18.637212 30189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:18.637441 30189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:18.639643 30039 heartbeater.cc:503] Master 127.21.27.190:37563 was elected leader, sending a full tablet report...
I20260809 22:44:18.640228 30189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:18.640345 30189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:18.640478 30189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:18.640554 30189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:18.665593 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.640796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:47339 (local address 127.21.27.190:37563)
0809 22:44:18.640963 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:18.640969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:18.642703 (+ 1734us) server_negotiation.cc:407] Connection header received
0809 22:44:18.645031 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:18.645037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:18.645123 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:18.645251 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:18.647406 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.648413 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.651330 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.651928 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.652091 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.657556 (+ 5465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:18.657561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:18.657564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:18.657598 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:18.660782 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.662017 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.662024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.662029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.662179 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.662849 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.662854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.662858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.663343 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:18.663526 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:18.664693 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0809 22:44:18.665040 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:44:18.666153 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.641631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.131:47339)
0809 22:44:18.642500 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:18.642551 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:18.644708 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:18.645474 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:18.645482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:18.646288 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:18.647183 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.647210 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.648605 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.648617 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.651068 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.651080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.652269 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.652281 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.652582 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.653538 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:18.653585 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:18.657304 (+ 3719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:18.660969 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.660991 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.661013 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.661855 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.662305 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.662314 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.662320 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.662690 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.663500 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:18.663509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:18.664093 (+ 584us) client_negotiation.cc:769] Sending connection context
0809 22:44:18.664485 (+ 392us) client_negotiation.cc:240] Negotiation successful
0809 22:44:18.664956 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":362,"threads_started":1}
I20260809 22:44:18.691466 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.672158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.21.27.131:42565)
0809 22:44:18.675694 (+ 3536us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:18.675716 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:18.675834 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:18.676241 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:18.676244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:18.676679 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:18.676977 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.676985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.678746 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.678754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.681148 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:18.681170 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.682030 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.682040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:18.682286 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.683140 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:18.683199 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:18.684372 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:18.687565 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.687571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.687575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.688204 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.688823 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:18.688830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:18.688836 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:18.689029 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:18.689762 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:18.689768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:18.690111 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 22:44:18.690291 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 22:44:18.690747 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3390,"thread_start_us":3159,"threads_started":1}
I20260809 22:44:18.691516 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:18.672541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:42565 (local address 127.21.27.190:37563)
0809 22:44:18.672764 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:18.672772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:18.675812 (+ 3040us) server_negotiation.cc:407] Connection header received
0809 22:44:18.675959 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:18.675969 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:18.676078 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:18.676206 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:18.677124 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.678478 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.681333 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.681795 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.681908 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:18.684588 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:18.684593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:18.684597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:18.684630 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:18.687424 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.688409 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.688414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.688416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.688588 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:18.689162 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:18.689168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:18.689172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:18.689627 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:18.689806 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:18.690805 (+ 999us) server_negotiation.cc:299] Negotiation successful
0809 22:44:18.691065 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:44:18.704607 30189 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:35237
I20260809 22:44:18.704630 30330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:35237 every 8 connection(s)
I20260809 22:44:18.705910 30189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:18.708206 30331 heartbeater.cc:347] Connected to a master server at 127.21.27.190:37563
I20260809 22:44:18.708456 30331 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:18.709106 30331 heartbeater.cc:511] Master 127.21.27.190:37563 requested a full tablet report, sending...
I20260809 22:44:18.710263 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30189
I20260809 22:44:18.710435 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:18.710608 29816 ts_manager.cc:194] Registered new tserver with Master: 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:18.710974 21614 external_mini_cluster.cc:1428] Setting key 1699adcdeddc16ba740d195fbce277c2
I20260809 22:44:18.711733 29816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:47339
I20260809 22:44:18.714304 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:18 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:19.172911 30185 heartbeater.cc:503] Master 127.21.27.190:37563 was elected leader, sending a full tablet report...
I20260809 22:44:19.715484 30331 heartbeater.cc:503] Master 127.21.27.190:37563 was elected leader, sending a full tablet report...
W20260809 22:44:19.868191 30340 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.101s user 0.006s sys 0.013s
W20260809 22:44:19.869477 29918 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:18.761261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.21.27.190:37563)
0809 22:44:18.761444 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:18.761448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:18.761732 (+ 284us) server_negotiation.cc:407] Connection header received
0809 22:44:18.761887 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:18.761892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:18.761957 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:18.762064 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:18.763487 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.764219 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.765421 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:18.765841 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:18.765942 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.869167 (+1103225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60592: BlockingRecv error: recv got EOF from 127.0.0.1:60592 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 22:44:19 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:19.886798 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.873256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.21.27.190:37563)
0809 22:44:19.873485 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:19.873492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:19.873762 (+ 270us) server_negotiation.cc:407] Connection header received
0809 22:44:19.873893 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:19.873897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:19.873994 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:19.874084 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:19.875363 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.876078 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.877264 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.877692 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.877785 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.881864 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:19.881872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:19.881879 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:19.881929 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:19.884578 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:19.885179 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:19.885182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:19.885185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:19.885301 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:19.885606 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:19.885609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:19.885611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:19.885850 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:19.886093 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:19.886321 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:44:19.886553 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:44:19.890838 29815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:19.898007 29815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:19.921311 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.909772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40473 (local address 127.0.0.1:33460)
0809 22:44:19.910814 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:19.910862 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:19.911090 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:19.913265 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:19.913275 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:19.913308 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:19.913718 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.913731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.916073 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.916078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.918690 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.918703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.920273 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.920279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.920468 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.920518 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 22:44:19.920659 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:44:19.920865 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":163,"thread_start_us":1014,"threads_started":1}
I20260809 22:44:19.921926 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.909922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39783 (local address 127.0.0.1:58140)
0809 22:44:19.911901 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:19.911925 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:19.912063 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:19.914296 (+ 2233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:19.914303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:19.914328 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:19.914701 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.914713 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.916454 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.916466 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.918695 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.918712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.920768 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.920774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.920922 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.920951 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:44:19.921049 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:44:19.921171 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1789,"mutex_wait_us":54,"thread_start_us":1578,"threads_started":1}
I20260809 22:44:19.922896 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.910618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.21.27.129:39783)
0809 22:44:19.913514 (+ 2896us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:19.913523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:19.913556 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:44:19.913668 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:19.913676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:19.913894 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:19.914077 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:19.914891 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.916327 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.918887 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.920296 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.920427 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.920616 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:19.922291 (+ 1675us) server_negotiation.cc:299] Negotiation successful
0809 22:44:19.922536 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2734,"thread_start_us":2591,"threads_started":1}
I20260809 22:44:19.923880 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.912461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.21.27.131:35237)
0809 22:44:19.914418 (+ 1957us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:19.914425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:19.914453 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:44:19.914557 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:19.914567 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:19.914775 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:19.914978 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:19.915843 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.917468 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.919989 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.921347 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.921497 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.923311 (+ 1814us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:19.923396 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 22:44:19.923572 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1756,"thread_start_us":1593,"threads_started":1}
I20260809 22:44:19.921202 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.909987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.21.27.130:40473)
0809 22:44:19.912574 (+ 2587us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:19.912586 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:19.912619 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:44:19.912753 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:19.912760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:19.913010 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:19.913934 (+ 924us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:19.914170 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.915878 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.918886 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.920046 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.920207 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.920416 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:19.920721 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 22:44:19.920890 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2429,"thread_start_us":2309,"threads_started":1}
I20260809 22:44:19.924556 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:19.911929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35237 (local address 127.0.0.1:36426)
0809 22:44:19.912861 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:19.912877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:19.912980 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:19.915086 (+ 2106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:19.915090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:19.915112 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:19.915523 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.915576 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.917617 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.917622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.919805 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:19.919823 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:19.921813 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:19.921823 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:19.922054 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:19.922100 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 22:44:19.922189 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:44:19.924191 (+ 2002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":2,"thread_start_us":792,"threads_started":1}
I20260809 22:44:19.927282 29964 tablet_service.cc:1467] Processing CreateTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 (DEFAULT_TABLE table=test-table [id=d55dd29abcfc435ea11da535e97b4477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:19.927867 30259 tablet_service.cc:1467] Processing CreateTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 (DEFAULT_TABLE table=test-table [id=d55dd29abcfc435ea11da535e97b4477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:19.927867 30111 tablet_service.cc:1467] Processing CreateTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 (DEFAULT_TABLE table=test-table [id=d55dd29abcfc435ea11da535e97b4477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:19.928792 29964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc78442054b4a3c8a823c8c0e4e3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:19.929171 30111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc78442054b4a3c8a823c8c0e4e3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:19.929179 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abc78442054b4a3c8a823c8c0e4e3609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:19.939697 30359 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Bootstrap starting.
I20260809 22:44:19.943939 30360 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Bootstrap starting.
I20260809 22:44:19.944864 30359 tablet_bootstrap.cc:654] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:19.945577 30361 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Bootstrap starting.
I20260809 22:44:19.946311 30359 log.cc:824] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:19.947628 30360 tablet_bootstrap.cc:654] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:19.948870 30361 tablet_bootstrap.cc:654] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:19.948863 30360 log.cc:824] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:19.950040 30361 log.cc:824] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:19.950356 30359 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: No bootstrap required, opened a new log
I20260809 22:44:19.950783 30359 ts_tablet_manager.cc:1397] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 22:44:19.952747 30361 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: No bootstrap required, opened a new log
I20260809 22:44:19.953006 30361 ts_tablet_manager.cc:1397] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 22:44:19.957792 30360 tablet_bootstrap.cc:492] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: No bootstrap required, opened a new log
I20260809 22:44:19.958078 30360 ts_tablet_manager.cc:1397] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260809 22:44:19.959430 30359 raft_consensus.cc:348] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.959916 30359 raft_consensus.cc:374] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:19.960091 30359 raft_consensus.cc:729] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Initialized, Role: FOLLOWER
I20260809 22:44:19.960484 30361 raft_consensus.cc:348] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.960976 30361 raft_consensus.cc:374] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:19.960929 30359 consensus_queue.cc:260] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.961144 30361 raft_consensus.cc:729] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Initialized, Role: FOLLOWER
I20260809 22:44:19.961910 30361 consensus_queue.cc:260] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.962888 30359 ts_tablet_manager.cc:1428] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260809 22:44:19.964483 30361 ts_tablet_manager.cc:1428] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260809 22:44:19.968928 30360 raft_consensus.cc:348] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.969353 30360 raft_consensus.cc:374] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:19.969434 30360 raft_consensus.cc:729] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Initialized, Role: FOLLOWER
I20260809 22:44:19.969938 30360 consensus_queue.cc:260] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:19.971624 30360 ts_tablet_manager.cc:1428] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 22:44:20.082690 30367 raft_consensus.cc:482] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:20.082989 30367 raft_consensus.cc:504] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.084576 30367 leader_election.cc:288] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:20.091555 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.085085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:46079 (local address 127.21.27.129:39783)
0809 22:44:20.085292 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.085297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.085747 (+ 450us) server_negotiation.cc:407] Connection header received
0809 22:44:20.085995 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.086002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.086096 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.086170 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.086931 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.087739 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.089552 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.090475 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.090602 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.090655 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.091101 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.091284 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:44:20.091572 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.084867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39783 (local address 127.21.27.130:46079)
0809 22:44:20.085576 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.085597 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.085752 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.086353 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.086357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.086379 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.086746 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.086762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.087884 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.087887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.089379 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.089391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.090705 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.090711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.090899 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.090947 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.091045 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.091239 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":354,"threads_started":1}
I20260809 22:44:20.091902 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.085805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:50159 (local address 127.21.27.131:35237)
0809 22:44:20.086046 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.086053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.086388 (+ 335us) server_negotiation.cc:407] Connection header received
0809 22:44:20.086611 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.086616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.086690 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.086786 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.087378 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.088066 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.089958 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.090878 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.091050 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.091117 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.091505 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.091656 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:44:20.092013 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.085661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35237 (local address 127.21.27.130:50159)
0809 22:44:20.086265 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.086279 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.086452 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.086816 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.086821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.086847 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.087226 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.087236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.088207 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.088215 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.089791 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.089805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.091128 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.091134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.091324 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.091361 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.091457 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.091595 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":2,"thread_start_us":292,"threads_started":1}
I20260809 22:44:20.092914 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc78442054b4a3c8a823c8c0e4e3609" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581" is_pre_election: true
I20260809 22:44:20.093078 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc78442054b4a3c8a823c8c0e4e3609" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" is_pre_election: true
I20260809 22:44:20.093562 29991 raft_consensus.cc:2454] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 0.
I20260809 22:44:20.093698 30286 raft_consensus.cc:2454] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 0.
I20260809 22:44:20.094314 30061 leader_election.cc:302] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eb880515d4ea08b50eb9cf18f19c2, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:20.094748 30367 raft_consensus.cc:2781] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:20.094928 30367 raft_consensus.cc:482] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:20.095055 30367 raft_consensus.cc:3037] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.098747 30367 raft_consensus.cc:504] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.099572 30367 leader_election.cc:288] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 election: Requested vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:20.099964 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc78442054b4a3c8a823c8c0e4e3609" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581"
I20260809 22:44:20.100124 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abc78442054b4a3c8a823c8c0e4e3609" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
I20260809 22:44:20.100188 29991 raft_consensus.cc:3037] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.100342 30286 raft_consensus.cc:3037] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.103390 30286 raft_consensus.cc:2454] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 1.
I20260809 22:44:20.103600 29991 raft_consensus.cc:2454] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 1.
I20260809 22:44:20.103788 30061 leader_election.cc:302] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eb880515d4ea08b50eb9cf18f19c2, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:20.104152 30367 raft_consensus.cc:2781] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:20.104614 30367 raft_consensus.cc:686] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 1 LEADER]: Becoming Leader. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Running, Role: LEADER
I20260809 22:44:20.105114 30367 consensus_queue.cc:237] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.109499 29816 catalog_manager.cc:5495] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 360667bc032e486590d876fd7e97afbd (127.21.27.130). New cstate: current_term: 1 leader_uuid: "360667bc032e486590d876fd7e97afbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } health_report { overall_health: UNKNOWN } } }
W20260809 22:44:20.137878 30040 tablet.cc:2369] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:20.159360 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.154498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.21.27.130:40473)
0809 22:44:20.154696 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.154700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.154718 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:20.154885 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.154888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.154988 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.155119 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:20.155493 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.156174 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.157403 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.157810 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.157899 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.158274 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.158817 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.158921 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.159019 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.159085 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:44:20.168191 30286 raft_consensus.cc:1261] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Refusing update from remote peer 360667bc032e486590d876fd7e97afbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:20.168200 29991 raft_consensus.cc:1261] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Refusing update from remote peer 360667bc032e486590d876fd7e97afbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:20.169268 30372 consensus_queue.cc:1035] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:20.169661 30367 consensus_queue.cc:1035] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:44:20.173100 30186 tablet.cc:2369] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:20.197949 29815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:20.198261 29815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:20.202158 29815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Sending DeleteTablet for 3 replicas of tablet abc78442054b4a3c8a823c8c0e4e3609
I20260809 22:44:20.203392 30111 tablet_service.cc:1514] Processing DeleteTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:33460
I20260809 22:44:20.203544 30259 tablet_service.cc:1514] Processing DeleteTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 22:44:20.203544 29964 tablet_service.cc:1514] Processing DeleteTablet for tablet abc78442054b4a3c8a823c8c0e4e3609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58140
I20260809 22:44:20.205154 29815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:20.206210 29815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:20.215585 30110 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9605814d842a29c680ded838883ca (DEFAULT_TABLE table=test-table [id=fb9297352ff3445fbac89e297a7a983c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:20.216161 29963 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9605814d842a29c680ded838883ca (DEFAULT_TABLE table=test-table [id=fb9297352ff3445fbac89e297a7a983c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:20.216521 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9605814d842a29c680ded838883ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.217983 30258 tablet_service.cc:1467] Processing CreateTablet for tablet b9c9605814d842a29c680ded838883ca (DEFAULT_TABLE table=test-table [id=fb9297352ff3445fbac89e297a7a983c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:20.218705 30258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9605814d842a29c680ded838883ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.223357 29963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c9605814d842a29c680ded838883ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.224308 30389 tablet_replica.cc:330] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: stopping tablet replica
I20260809 22:44:20.224956 30389 raft_consensus.cc:2229] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:20.226815 30361 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Bootstrap starting.
I20260809 22:44:20.228783 30361 tablet_bootstrap.cc:654] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.228914 30389 raft_consensus.cc:2258] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.231534 30389 ts_tablet_manager.cc:1905] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:20.235773 30391 tablet_replica.cc:330] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: stopping tablet replica
I20260809 22:44:20.236317 30391 raft_consensus.cc:2229] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:20.236990 30391 raft_consensus.cc:2258] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.238159 30359 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Bootstrap starting.
I20260809 22:44:20.239096 30360 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Bootstrap starting.
I20260809 22:44:20.239959 30359 tablet_bootstrap.cc:654] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.240778 30360 tablet_bootstrap.cc:654] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.243616 30390 tablet_replica.cc:330] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: stopping tablet replica
I20260809 22:44:20.244199 30390 raft_consensus.cc:2229] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:20.244778 30390 raft_consensus.cc:2258] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.247927 30391 ts_tablet_manager.cc:1905] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:20.252933 30390 ts_tablet_manager.cc:1905] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:20.254081 30361 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: No bootstrap required, opened a new log
I20260809 22:44:20.254252 30359 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: No bootstrap required, opened a new log
I20260809 22:44:20.254386 30361 ts_tablet_manager.cc:1397] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.012s
I20260809 22:44:20.254441 30359 ts_tablet_manager.cc:1397] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260809 22:44:20.255455 30359 raft_consensus.cc:348] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.255705 30359 raft_consensus.cc:374] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.255803 30359 raft_consensus.cc:729] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.256858 30360 tablet_bootstrap.cc:492] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: No bootstrap required, opened a new log
I20260809 22:44:20.257025 30360 ts_tablet_manager.cc:1397] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.013s
I20260809 22:44:20.257206 30361 raft_consensus.cc:348] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.257547 30361 raft_consensus.cc:374] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.257702 30361 raft_consensus.cc:729] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.257845 30360 raft_consensus.cc:348] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.258082 30360 raft_consensus.cc:374] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.258193 30360 raft_consensus.cc:729] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.258127 30361 consensus_queue.cc:260] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.258579 30359 consensus_queue.cc:260] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.260421 30390 ts_tablet_manager.cc:1918] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:20.260607 30390 log.cc:1196] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/abc78442054b4a3c8a823c8c0e4e3609
I20260809 22:44:20.260931 30359 ts_tablet_manager.cc:1428] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260809 22:44:20.261296 30390 ts_tablet_manager.cc:1939] T abc78442054b4a3c8a823c8c0e4e3609 P 0d001893dbd84c908be23f54d4e7e581: Deleting consensus metadata
I20260809 22:44:20.261307 30360 consensus_queue.cc:260] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.263342 29799 catalog_manager.cc:4841] TS 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783): tablet abc78442054b4a3c8a823c8c0e4e3609 (table test-table [id=d55dd29abcfc435ea11da535e97b4477]) successfully deleted
I20260809 22:44:20.263861 30360 ts_tablet_manager.cc:1428] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent starting tablet: real 0.007s user 0.001s sys 0.002s
I20260809 22:44:20.264858 30361 ts_tablet_manager.cc:1428] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260809 22:44:20.266548 30389 ts_tablet_manager.cc:1918] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:20.266773 30389 log.cc:1196] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/abc78442054b4a3c8a823c8c0e4e3609
I20260809 22:44:20.267479 30389 ts_tablet_manager.cc:1939] T abc78442054b4a3c8a823c8c0e4e3609 P 360667bc032e486590d876fd7e97afbd: Deleting consensus metadata
I20260809 22:44:20.269047 29802 catalog_manager.cc:4841] TS 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473): tablet abc78442054b4a3c8a823c8c0e4e3609 (table test-table [id=d55dd29abcfc435ea11da535e97b4477]) successfully deleted
I20260809 22:44:20.269629 30391 ts_tablet_manager.cc:1918] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:20.269816 30391 log.cc:1196] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/abc78442054b4a3c8a823c8c0e4e3609
I20260809 22:44:20.270278 30391 ts_tablet_manager.cc:1939] T abc78442054b4a3c8a823c8c0e4e3609 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting consensus metadata
I20260809 22:44:20.271925 29799 catalog_manager.cc:4841] TS 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237): tablet abc78442054b4a3c8a823c8c0e4e3609 (table test-table [id=d55dd29abcfc435ea11da535e97b4477]) successfully deleted
I20260809 22:44:20.449962 30368 raft_consensus.cc:482] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:20.450196 30368 raft_consensus.cc:504] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.452095 30368 leader_election.cc:288] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473)
W20260809 22:44:20.459575 30332 tablet.cc:2369] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:20.461041 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.452743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:45747 (local address 127.21.27.130:40473)
0809 22:44:20.452996 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.453005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.453883 (+ 878us) server_negotiation.cc:407] Connection header received
0809 22:44:20.454024 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.454029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.454138 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.454254 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.455060 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.456300 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.458681 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.459773 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.459948 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.460046 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.460544 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.460700 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:44:20.461059 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.452885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:47053 (local address 127.21.27.129:39783)
0809 22:44:20.453086 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.453094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.453670 (+ 576us) server_negotiation.cc:407] Connection header received
0809 22:44:20.453986 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.453995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.454083 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.454165 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.455120 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.455914 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.458529 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.459872 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.460064 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.460150 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.460633 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.460779 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:44:20.461160 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.452571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40473 (local address 127.21.27.131:45747)
0809 22:44:20.453760 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.453776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.453881 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.454309 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.454317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.454348 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.454847 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.454868 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.456442 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.456447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.458503 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.458521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.460104 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.460110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.460337 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.460373 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.460479 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.460650 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":1,"spinlock_wait_cycles":9472,"thread_start_us":606,"threads_started":1}
I20260809 22:44:20.461905 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.452393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39783 (local address 127.21.27.131:47053)
0809 22:44:20.453524 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.453549 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.453801 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.454398 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.454407 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.454436 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.454885 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.454899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.456060 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.456068 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.458333 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.458351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.460267 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.460273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.460430 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.460471 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.461457 (+ 986us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.461597 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"spinlock_wait_cycles":768,"thread_start_us":1111,"threads_started":1}
I20260809 22:44:20.462455 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9605814d842a29c680ded838883ca" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd" is_pre_election: true
I20260809 22:44:20.462769 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9605814d842a29c680ded838883ca" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581" is_pre_election: true
I20260809 22:44:20.463053 29991 raft_consensus.cc:2454] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 0.
I20260809 22:44:20.463024 30134 raft_consensus.cc:2454] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 0.
I20260809 22:44:20.463634 30208 leader_election.cc:302] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 1e6eb880515d4ea08b50eb9cf18f19c2; no voters:
I20260809 22:44:20.464062 30368 raft_consensus.cc:2781] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:20.464208 30368 raft_consensus.cc:482] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:20.464335 30368 raft_consensus.cc:3037] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.466789 30368 raft_consensus.cc:504] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.467736 30368 leader_election.cc:288] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 election: Requested vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473)
I20260809 22:44:20.468350 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9605814d842a29c680ded838883ca" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581"
I20260809 22:44:20.468415 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9c9605814d842a29c680ded838883ca" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd"
I20260809 22:44:20.468577 29991 raft_consensus.cc:3037] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.468657 30134 raft_consensus.cc:3037] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.471061 29991 raft_consensus.cc:2454] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 1.
I20260809 22:44:20.472359 30134 raft_consensus.cc:2454] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 1.
I20260809 22:44:20.472877 30210 leader_election.cc:302] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eb880515d4ea08b50eb9cf18f19c2, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:20.473229 30368 raft_consensus.cc:2781] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:20.473771 30368 raft_consensus.cc:686] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Running, Role: LEADER
I20260809 22:44:20.474169 30368 consensus_queue.cc:237] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.477900 29816 catalog_manager.cc:5495] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } health_report { overall_health: HEALTHY } } }
Aug 09 22:44:20 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315456, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.190@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:20.574822 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.557702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:37563 (local address 127.0.0.1:60610)
0809 22:44:20.558510 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.558544 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.558743 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.559177 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.559182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.560224 (+ 1042us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:20.560490 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.560505 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.561640 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.561645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.563285 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.563303 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.564056 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.564070 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.564261 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.565052 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:20.565080 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:20.567920 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:20.572115 (+ 4195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:20.572128 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:20.572134 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:20.572601 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:20.573076 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:20.573080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:20.573082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:20.573246 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:20.573979 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:20.573986 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:20.574149 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.574299 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.574508 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":403,"threads_started":1}
I20260809 22:44:20.575245 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.557777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.21.27.190:37563)
0809 22:44:20.558830 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.558835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.558856 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:44:20.558902 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.558906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.558961 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.559056 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:20.560682 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.561468 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.563450 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.563867 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.563959 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.568129 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:20.568138 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:20.568143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:20.568182 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:20.571945 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:20.572745 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:20.572753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:20.572759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:20.572947 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:20.573389 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:20.573395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:20.573398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:20.573838 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:20.574004 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.574464 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.574738 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":807,"threads_started":1}
I20260809 22:44:20.581244 29816 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:60610:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:44:20.582659 29816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:20.590971 30110 tablet_service.cc:1467] Processing CreateTablet for tablet 9861c1aff73a4cdb829269300ac52700 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3465ac1067c341aabc4943119009fc46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:44:20.591069 29963 tablet_service.cc:1467] Processing CreateTablet for tablet 9861c1aff73a4cdb829269300ac52700 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3465ac1067c341aabc4943119009fc46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:44:20.591194 30258 tablet_service.cc:1467] Processing CreateTablet for tablet 9861c1aff73a4cdb829269300ac52700 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3465ac1067c341aabc4943119009fc46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:44:20.591632 29963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9861c1aff73a4cdb829269300ac52700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.591638 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9861c1aff73a4cdb829269300ac52700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.591780 30258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9861c1aff73a4cdb829269300ac52700. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:20.599814 30359 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: Bootstrap starting.
I20260809 22:44:20.600710 30361 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: Bootstrap starting.
I20260809 22:44:20.600884 30360 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: Bootstrap starting.
I20260809 22:44:20.601719 30359 tablet_bootstrap.cc:654] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.602568 30361 tablet_bootstrap.cc:654] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.602576 30360 tablet_bootstrap.cc:654] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:20.615988 30361 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: No bootstrap required, opened a new log
I20260809 22:44:20.616324 30361 ts_tablet_manager.cc:1397] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.009s
I20260809 22:44:20.616511 30359 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: No bootstrap required, opened a new log
I20260809 22:44:20.616736 30359 ts_tablet_manager.cc:1397] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.011s
I20260809 22:44:20.617058 30360 tablet_bootstrap.cc:492] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: No bootstrap required, opened a new log
I20260809 22:44:20.617331 30360 ts_tablet_manager.cc:1397] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260809 22:44:20.617610 30359 raft_consensus.cc:348] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.617898 30359 raft_consensus.cc:374] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.617684 30361 raft_consensus.cc:348] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.618005 30359 raft_consensus.cc:729] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.618228 30361 raft_consensus.cc:374] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.618100 30360 raft_consensus.cc:348] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.618352 30360 raft_consensus.cc:374] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:20.618422 30360 raft_consensus.cc:729] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.618314 30359 consensus_queue.cc:260] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.618716 30360 consensus_queue.cc:260] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.618815 30366 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.619247 30366 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.619123 30361 raft_consensus.cc:729] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Initialized, Role: FOLLOWER
I20260809 22:44:20.619129 30368 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.619451 30368 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.619697 30361 consensus_queue.cc:260] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.620260 30367 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.620568 30367 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.621131 30360 ts_tablet_manager.cc:1428] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 22:44:20.621353 30361 ts_tablet_manager.cc:1428] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 22:44:20.622540 30359 ts_tablet_manager.cc:1428] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260809 22:44:20.745203 30366 raft_consensus.cc:482] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:20.745427 30366 raft_consensus.cc:504] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.746879 30366 leader_election.cc:288] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:20.755570 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.747737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48819 (local address 127.21.27.130:40473)
0809 22:44:20.747937 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.747943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.748482 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:44:20.748649 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.748656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.748743 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.748835 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.749447 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.750457 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.753624 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.754467 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.754669 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.754747 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.755096 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.755281 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:44:20.755754 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.747286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40473 (local address 127.21.27.129:48819)
0809 22:44:20.748367 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.748381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.748476 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.748888 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.748894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.748925 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.749286 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.749295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.750566 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.750570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.753395 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.753413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.754691 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.754698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.754881 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.754937 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.755068 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.755256 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":250,"thread_start_us":893,"threads_started":1}
I20260809 22:44:20.755741 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.747421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:38155 (local address 127.21.27.131:35237)
0809 22:44:20.748190 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.748197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.748221 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:44:20.748301 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.748306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.748391 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.748527 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:20.749206 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.750390 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.753597 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.754638 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.754850 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.754930 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.755284 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.755431 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":521,"threads_started":1}
I20260809 22:44:20.755861 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.747249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35237 (local address 127.21.27.129:38155)
0809 22:44:20.747816 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.747838 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.747991 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.748572 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.748577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.748603 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:20.748991 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.749010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.750558 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.750562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.753395 (+ 2833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.753413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.754886 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.754892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.755087 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.755111 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.755257 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.755413 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":201,"thread_start_us":281,"threads_started":1}
I20260809 22:44:20.756610 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9861c1aff73a4cdb829269300ac52700" candidate_uuid: "0d001893dbd84c908be23f54d4e7e581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd" is_pre_election: true
I20260809 22:44:20.756762 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9861c1aff73a4cdb829269300ac52700" candidate_uuid: "0d001893dbd84c908be23f54d4e7e581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" is_pre_election: true
I20260809 22:44:20.756898 30134 raft_consensus.cc:2454] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d001893dbd84c908be23f54d4e7e581 in term 0.
I20260809 22:44:20.757035 30286 raft_consensus.cc:2454] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d001893dbd84c908be23f54d4e7e581 in term 0.
I20260809 22:44:20.757541 29913 leader_election.cc:302] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 1e6eb880515d4ea08b50eb9cf18f19c2, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:20.757995 30366 raft_consensus.cc:2781] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:20.758109 30366 raft_consensus.cc:482] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:20.758248 30366 raft_consensus.cc:3037] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.760571 30366 raft_consensus.cc:504] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.761281 30366 leader_election.cc:288] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [CANDIDATE]: Term 1 election: Requested vote from peers 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:20.761826 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9861c1aff73a4cdb829269300ac52700" candidate_uuid: "0d001893dbd84c908be23f54d4e7e581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd"
I20260809 22:44:20.762092 30134 raft_consensus.cc:3037] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.762077 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9861c1aff73a4cdb829269300ac52700" candidate_uuid: "0d001893dbd84c908be23f54d4e7e581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
I20260809 22:44:20.762328 30286 raft_consensus.cc:3037] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:20.764307 30134 raft_consensus.cc:2454] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d001893dbd84c908be23f54d4e7e581 in term 1.
I20260809 22:44:20.764585 30286 raft_consensus.cc:2454] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d001893dbd84c908be23f54d4e7e581 in term 1.
I20260809 22:44:20.764828 29913 leader_election.cc:302] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:20.765141 30366 raft_consensus.cc:2781] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:20.765483 30366 raft_consensus.cc:686] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [term 1 LEADER]: Becoming Leader. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Running, Role: LEADER
I20260809 22:44:20.765904 30366 consensus_queue.cc:237] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:20.767923 30414 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: TxnStatusTablet state changed. Reason: New leader 0d001893dbd84c908be23f54d4e7e581. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.768399 30414 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:44:20.769090 30416 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:44:20.769342 30416 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:44:20.769857 29816 catalog_manager.cc:5495] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } health_report { overall_health: UNKNOWN } } }
I20260809 22:44:20.813546 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.807259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.21.27.129:39783)
0809 22:44:20.807442 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.807447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.807465 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:20.807751 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.807756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.807834 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.807948 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:20.808396 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.809142 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.811073 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.811592 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.811716 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.812334 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.812883 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.812993 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.813150 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.813225 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:20.813638 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.807051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39783 (local address 127.0.0.1:58152)
0809 22:44:20.807362 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.807383 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.807530 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.808010 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.808015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.808029 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:44:20.808240 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.808251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.809294 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.809299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.810906 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.810921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.811770 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.811778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.812039 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.812121 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.813000 (+ 879us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.813026 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.813123 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.813310 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 22:44:20.818637 30286 raft_consensus.cc:1261] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Refusing update from remote peer 0d001893dbd84c908be23f54d4e7e581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:20.818773 30134 raft_consensus.cc:1261] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Refusing update from remote peer 0d001893dbd84c908be23f54d4e7e581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:20.819433 30414 consensus_queue.cc:1035] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:20.819765 30366 consensus_queue.cc:1035] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:20.831445 30367 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: TxnStatusTablet state changed. Reason: New leader 0d001893dbd84c908be23f54d4e7e581. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.831808 30367 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.834332 30414 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.834892 30421 mvcc.cc:205] Tried to move back new op lower bound from 7316748127505772544 to 7316748127300931584. Current Snapshot: MvccSnapshot[applied={T|T < 7316748127505772544}]
I20260809 22:44:20.835382 30368 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: TxnStatusTablet state changed. Reason: New leader 0d001893dbd84c908be23f54d4e7e581. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.835707 30368 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.837553 30414 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:44:20.838240 30414 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.838498 30414 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 0d001893dbd84c908be23f54d4e7e581: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:44:20.839857 30367 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.840111 30367 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 360667bc032e486590d876fd7e97afbd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.844686 30368 tablet_replica.cc:439] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d001893dbd84c908be23f54d4e7e581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } } }
I20260809 22:44:20.845012 30368 tablet_replica.cc:442] T 9861c1aff73a4cdb829269300ac52700 P 1e6eb880515d4ea08b50eb9cf18f19c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:44:20.854324 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.848066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.21.27.131:35237)
0809 22:44:20.848283 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.848293 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.848757 (+ 464us) server_negotiation.cc:407] Connection header received
0809 22:44:20.848989 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.848992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.849070 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.849239 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:20.849699 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.850503 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.851897 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.852455 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.852584 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.853126 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.853663 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.853790 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.853946 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.854019 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:20.859546 30368 consensus_queue.cc:1035] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:20.867527 30368 consensus_queue.cc:1035] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:20.878296 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.868366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:48865 (local address 127.21.27.129:39783)
0809 22:44:20.868926 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.868936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.871906 (+ 2970us) server_negotiation.cc:407] Connection header received
0809 22:44:20.872197 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.872203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.872293 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.872358 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:20.873161 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.874280 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.876429 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.876948 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.877049 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.877613 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.877736 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.877804 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.877969 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.878041 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260809 22:44:20.878515 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.868212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39783 (local address 127.21.27.131:48865)
0809 22:44:20.871790 (+ 3578us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.871817 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.872028 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.872552 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.872557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.872571 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:44:20.872970 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.872982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.874449 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.874456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.876274 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.876294 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.877141 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.877148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.877337 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.877428 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.877839 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.877852 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.877940 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.878091 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3394,"thread_start_us":3175,"threads_started":1}
I20260809 22:44:20.923779 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.915903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:40277 (local address 127.21.27.131:35237)
0809 22:44:20.916130 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:20.916137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:20.917367 (+ 1230us) server_negotiation.cc:407] Connection header received
0809 22:44:20.917587 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:20.917594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:20.917709 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:20.917809 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:20.918776 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.919589 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.921758 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.922333 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.922433 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.923038 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.923230 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.923316 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:20.923482 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 22:44:20.923553 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:20.924058 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:20.915530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35237 (local address 127.21.27.129:40277)
0809 22:44:20.917267 (+ 1737us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:20.917283 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:20.917440 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:20.918207 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:20.918212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:20.918226 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:44:20.918630 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.918645 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.919718 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.919724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.921604 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:20.921619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:20.922526 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:20.922533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:20.922728 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:20.922805 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:44:20.923341 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:44:20.923359 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 22:44:20.923461 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:44:20.923620 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1537,"thread_start_us":1374,"threads_started":1}
I20260809 22:44:20.965256 29816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:20.965559 29816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:20.969449 29816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Sending DeleteTablet for 3 replicas of tablet b9c9605814d842a29c680ded838883ca
I20260809 22:44:20.970592 30258 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c9605814d842a29c680ded838883ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 22:44:20.971091 30110 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c9605814d842a29c680ded838883ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:33460
I20260809 22:44:20.971386 30443 tablet_replica.cc:330] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: stopping tablet replica
I20260809 22:44:20.971639 30443 raft_consensus.cc:2229] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:20.971728 29963 tablet_service.cc:1514] Processing DeleteTablet for tablet b9c9605814d842a29c680ded838883ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:20 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58140
I20260809 22:44:20.972348 30443 raft_consensus.cc:2258] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.979441 30445 tablet_replica.cc:330] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: stopping tablet replica
I20260809 22:44:20.979761 30445 raft_consensus.cc:2229] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:20.980212 30445 raft_consensus.cc:2258] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.981498 30445 ts_tablet_manager.cc:1905] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:20.988235 30443 ts_tablet_manager.cc:1905] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:20.988945 30445 ts_tablet_manager.cc:1918] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:20.989084 30445 log.cc:1196] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/b9c9605814d842a29c680ded838883ca
I20260809 22:44:20.989574 30445 ts_tablet_manager.cc:1939] T b9c9605814d842a29c680ded838883ca P 0d001893dbd84c908be23f54d4e7e581: Deleting consensus metadata
I20260809 22:44:20.990809 29799 catalog_manager.cc:4841] TS 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783): tablet b9c9605814d842a29c680ded838883ca (table test-table [id=fb9297352ff3445fbac89e297a7a983c]) successfully deleted
I20260809 22:44:20.996451 30444 tablet_replica.cc:330] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: stopping tablet replica
I20260809 22:44:20.996693 30444 raft_consensus.cc:2229] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:20.997012 30444 raft_consensus.cc:2258] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:20.997923 30443 ts_tablet_manager.cc:1918] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:20.998136 30443 log.cc:1196] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/b9c9605814d842a29c680ded838883ca
I20260809 22:44:20.998768 30443 ts_tablet_manager.cc:1939] T b9c9605814d842a29c680ded838883ca P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting consensus metadata
I20260809 22:44:21.000260 29799 catalog_manager.cc:4841] TS 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237): tablet b9c9605814d842a29c680ded838883ca (table test-table [id=fb9297352ff3445fbac89e297a7a983c]) successfully deleted
I20260809 22:44:21.004149 30444 ts_tablet_manager.cc:1905] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.012054 30444 ts_tablet_manager.cc:1918] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:21.012303 30444 log.cc:1196] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/b9c9605814d842a29c680ded838883ca
I20260809 22:44:21.012827 30444 ts_tablet_manager.cc:1939] T b9c9605814d842a29c680ded838883ca P 360667bc032e486590d876fd7e97afbd: Deleting consensus metadata
I20260809 22:44:21.014123 29802 catalog_manager.cc:4841] TS 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473): tablet b9c9605814d842a29c680ded838883ca (table test-table [id=fb9297352ff3445fbac89e297a7a983c]) successfully deleted
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:21.077256 30405 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:21.062727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.21.27.190:37563)
0809 22:44:21.062929 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.062935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.063547 (+ 612us) server_negotiation.cc:407] Connection header received
0809 22:44:21.063726 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.063732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.063867 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.063996 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:21.065753 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.066912 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.068938 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.069664 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.069802 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.076964 (+ 7162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60614: BlockingRecv error: recv got EOF from 127.0.0.1:60614 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM
I20260809 22:44:21.100353 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:21.081838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.21.27.190:37563)
0809 22:44:21.082057 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.082063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.082085 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:44:21.082256 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.082263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.082346 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.082421 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:21.084177 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.085315 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.087177 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.087848 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.087982 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.093185 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:21.093193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:21.093198 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:21.093244 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:21.096998 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.097907 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.097913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.097917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.098077 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.098630 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.098635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.098638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.099021 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:21.099523 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:21.099673 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 22:44:21.099926 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:21.104655 29816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:21.105403 29816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:21.113499 29963 tablet_service.cc:1467] Processing CreateTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 (DEFAULT_TABLE table=test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.113490 30110 tablet_service.cc:1467] Processing CreateTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 (DEFAULT_TABLE table=test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.114078 29963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e866ec1e549d48c697b1bd1d6d2a3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.114241 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e866ec1e549d48c697b1bd1d6d2a3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.120043 30258 tablet_service.cc:1467] Processing CreateTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 (DEFAULT_TABLE table=test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.120541 30258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e866ec1e549d48c697b1bd1d6d2a3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.123008 30359 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Bootstrap starting.
I20260809 22:44:21.125679 30359 tablet_bootstrap.cc:654] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.129892 30359 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: No bootstrap required, opened a new log
I20260809 22:44:21.130147 30359 ts_tablet_manager.cc:1397] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260809 22:44:21.131099 30359 raft_consensus.cc:348] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.132798 30359 raft_consensus.cc:374] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.132972 30359 raft_consensus.cc:729] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.133365 30359 consensus_queue.cc:260] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.134043 30359 ts_tablet_manager.cc:1428] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:44:21.137600 30461 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Bootstrap starting.
I20260809 22:44:21.141140 30461 tablet_bootstrap.cc:654] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.146286 30461 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: No bootstrap required, opened a new log
I20260809 22:44:21.146507 30461 ts_tablet_manager.cc:1397] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260809 22:44:21.147406 30461 raft_consensus.cc:348] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.147652 30462 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Bootstrap starting.
I20260809 22:44:21.147845 30461 raft_consensus.cc:374] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.148007 30461 raft_consensus.cc:729] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.148343 30461 consensus_queue.cc:260] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.149407 30462 tablet_bootstrap.cc:654] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.149562 30461 ts_tablet_manager.cc:1428] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:21.154968 30462 tablet_bootstrap.cc:492] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: No bootstrap required, opened a new log
I20260809 22:44:21.155114 30462 ts_tablet_manager.cc:1397] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260809 22:44:21.155959 30462 raft_consensus.cc:348] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.156213 30462 raft_consensus.cc:374] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.156311 30462 raft_consensus.cc:729] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.156595 30462 consensus_queue.cc:260] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.157194 30462 ts_tablet_manager.cc:1428] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 22:44:21.411486 30367 raft_consensus.cc:482] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:21.411707 30367 raft_consensus.cc:504] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.412396 30367 leader_election.cc:288] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:21.413120 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e866ec1e549d48c697b1bd1d6d2a3db1" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" is_pre_election: true
I20260809 22:44:21.413156 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e866ec1e549d48c697b1bd1d6d2a3db1" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581" is_pre_election: true
I20260809 22:44:21.413406 30286 raft_consensus.cc:2454] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 0.
I20260809 22:44:21.413461 29991 raft_consensus.cc:2454] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 0.
I20260809 22:44:21.414028 30061 leader_election.cc:302] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:21.414352 30367 raft_consensus.cc:2781] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:21.414487 30367 raft_consensus.cc:482] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:21.414585 30367 raft_consensus.cc:3037] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.416993 30367 raft_consensus.cc:504] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.417570 30367 leader_election.cc:288] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 election: Requested vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237)
I20260809 22:44:21.418085 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e866ec1e549d48c697b1bd1d6d2a3db1" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581"
I20260809 22:44:21.418176 30286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e866ec1e549d48c697b1bd1d6d2a3db1" candidate_uuid: "360667bc032e486590d876fd7e97afbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2"
I20260809 22:44:21.418313 29991 raft_consensus.cc:3037] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.418380 30286 raft_consensus.cc:3037] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.420322 29991 raft_consensus.cc:2454] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 1.
I20260809 22:44:21.420796 30061 leader_election.cc:302] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:21.421131 30367 raft_consensus.cc:2781] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:21.421603 30367 raft_consensus.cc:686] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 1 LEADER]: Becoming Leader. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Running, Role: LEADER
I20260809 22:44:21.421715 30286 raft_consensus.cc:2454] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 360667bc032e486590d876fd7e97afbd in term 1.
I20260809 22:44:21.421962 30367 consensus_queue.cc:237] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.425180 29814 catalog_manager.cc:5495] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 360667bc032e486590d876fd7e97afbd (127.21.27.130). New cstate: current_term: 1 leader_uuid: "360667bc032e486590d876fd7e97afbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } health_report { overall_health: UNKNOWN } } }
I20260809 22:44:21.505806 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:21.501129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.21.27.130:40473)
0809 22:44:21.501596 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.501601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.501618 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:44:21.501676 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.501682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.501743 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.501828 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:21.502162 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.502924 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.504290 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.504683 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.504776 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.505133 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:21.505231 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:21.505294 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:21.505417 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 22:44:21.505475 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":252,"threads_started":1}
I20260809 22:44:21.508742 29991 raft_consensus.cc:1261] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Refusing update from remote peer 360667bc032e486590d876fd7e97afbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:21.508837 30286 raft_consensus.cc:1261] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Refusing update from remote peer 360667bc032e486590d876fd7e97afbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:21.509459 30465 consensus_queue.cc:1035] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:21.509807 30367 consensus_queue.cc:1035] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:21.523593 29814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:21.523859 29814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:21.527717 29814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Sending DeleteTablet for 3 replicas of tablet e866ec1e549d48c697b1bd1d6d2a3db1
I20260809 22:44:21.528616 30258 tablet_service.cc:1514] Processing DeleteTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 22:44:21.528772 30259 tablet_service.cc:1514] Processing DeleteTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 22:44:21.528893 29963 tablet_service.cc:1514] Processing DeleteTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58140
I20260809 22:44:21.528964 30110 tablet_service.cc:1514] Processing DeleteTablet for tablet e866ec1e549d48c697b1bd1d6d2a3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:33460
I20260809 22:44:21.532059 29814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:21.532884 29814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:21.539906 30482 tablet_replica.cc:330] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: stopping tablet replica
I20260809 22:44:21.540103 30482 raft_consensus.cc:2229] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:21.540532 30482 raft_consensus.cc:2258] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.541607 30111 tablet_service.cc:1467] Processing CreateTablet for tablet a312e7b41d0c45788ebd85428064f74c (DEFAULT_TABLE table=test-table [id=d9b43157aecd4df087ccd724f9f60f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.542482 30111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a312e7b41d0c45788ebd85428064f74c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.543262 30480 tablet_replica.cc:330] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: stopping tablet replica
I20260809 22:44:21.543470 30480 raft_consensus.cc:2229] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:21.543627 29964 tablet_service.cc:1467] Processing CreateTablet for tablet a312e7b41d0c45788ebd85428064f74c (DEFAULT_TABLE table=test-table [id=d9b43157aecd4df087ccd724f9f60f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.543756 30480 raft_consensus.cc:2258] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.543841 30259 tablet_service.cc:1467] Processing CreateTablet for tablet a312e7b41d0c45788ebd85428064f74c (DEFAULT_TABLE table=test-table [id=d9b43157aecd4df087ccd724f9f60f63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:21.544236 29964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a312e7b41d0c45788ebd85428064f74c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.544310 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a312e7b41d0c45788ebd85428064f74c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:21.547905 30482 ts_tablet_manager.cc:1905] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.548072 30479 tablet_replica.cc:330] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: stopping tablet replica
I20260809 22:44:21.548285 30479 raft_consensus.cc:2229] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:21.548542 30479 raft_consensus.cc:2258] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.548549 30480 ts_tablet_manager.cc:1905] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.550148 30462 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Bootstrap starting.
I20260809 22:44:21.551878 30462 tablet_bootstrap.cc:654] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.554838 30482 ts_tablet_manager.cc:1918] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:21.554838 30480 ts_tablet_manager.cc:1918] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:21.554972 30482 log.cc:1196] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/e866ec1e549d48c697b1bd1d6d2a3db1
I20260809 22:44:21.554972 30480 log.cc:1196] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/e866ec1e549d48c697b1bd1d6d2a3db1
I20260809 22:44:21.555491 30480 ts_tablet_manager.cc:1939] T e866ec1e549d48c697b1bd1d6d2a3db1 P 0d001893dbd84c908be23f54d4e7e581: Deleting consensus metadata
I20260809 22:44:21.555610 30482 ts_tablet_manager.cc:1939] T e866ec1e549d48c697b1bd1d6d2a3db1 P 360667bc032e486590d876fd7e97afbd: Deleting consensus metadata
I20260809 22:44:21.556654 29802 catalog_manager.cc:4841] TS 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473): tablet e866ec1e549d48c697b1bd1d6d2a3db1 (table test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]) successfully deleted
I20260809 22:44:21.556893 29799 catalog_manager.cc:4841] TS 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783): tablet e866ec1e549d48c697b1bd1d6d2a3db1 (table test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]) successfully deleted
I20260809 22:44:21.557900 30462 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: No bootstrap required, opened a new log
I20260809 22:44:21.558104 30462 ts_tablet_manager.cc:1397] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:44:21.559298 30462 raft_consensus.cc:348] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.559638 30462 raft_consensus.cc:374] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.559659 30461 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Bootstrap starting.
I20260809 22:44:21.559803 30462 raft_consensus.cc:729] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.560194 30479 ts_tablet_manager.cc:1905] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.560133 30462 consensus_queue.cc:260] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.560824 30462 ts_tablet_manager.cc:1428] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:21.561280 30461 tablet_bootstrap.cc:654] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.562330 30359 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Bootstrap starting.
I20260809 22:44:21.564237 30359 tablet_bootstrap.cc:654] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:21.566408 30479 ts_tablet_manager.cc:1918] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:21.566548 30479 log.cc:1196] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/e866ec1e549d48c697b1bd1d6d2a3db1
I20260809 22:44:21.567049 30479 ts_tablet_manager.cc:1939] T e866ec1e549d48c697b1bd1d6d2a3db1 P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting consensus metadata
I20260809 22:44:21.567127 30359 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: No bootstrap required, opened a new log
I20260809 22:44:21.567346 30359 ts_tablet_manager.cc:1397] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 22:44:21.567890 30461 tablet_bootstrap.cc:492] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: No bootstrap required, opened a new log
I20260809 22:44:21.568075 30461 ts_tablet_manager.cc:1397] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260809 22:44:21.568166 29799 catalog_manager.cc:4841] TS 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237): tablet e866ec1e549d48c697b1bd1d6d2a3db1 (table test-table [id=e181b6ea9ef4459b8ab6b90e55d67d75]) successfully deleted
W20260809 22:44:21.568744 29799 catalog_manager.cc:4808] TS 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237): delete failed for tablet e866ec1e549d48c697b1bd1d6d2a3db1 because the tablet was not found. No further retry: Not found: Tablet not found: e866ec1e549d48c697b1bd1d6d2a3db1
I20260809 22:44:21.568619 30359 raft_consensus.cc:348] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.568951 30359 raft_consensus.cc:374] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.568820 30461 raft_consensus.cc:348] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.569046 30359 raft_consensus.cc:729] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d001893dbd84c908be23f54d4e7e581, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.569113 30461 raft_consensus.cc:374] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:21.569212 30461 raft_consensus.cc:729] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360667bc032e486590d876fd7e97afbd, State: Initialized, Role: FOLLOWER
I20260809 22:44:21.569283 30359 consensus_queue.cc:260] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.569446 30461 consensus_queue.cc:260] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.570039 30461 ts_tablet_manager.cc:1428] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:44:21.570276 30359 ts_tablet_manager.cc:1428] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:21.641230 30437 raft_consensus.cc:482] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:21.641458 30437 raft_consensus.cc:504] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.642154 30437 leader_election.cc:288] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473)
I20260809 22:44:21.642714 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a312e7b41d0c45788ebd85428064f74c" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581" is_pre_election: true
I20260809 22:44:21.642814 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a312e7b41d0c45788ebd85428064f74c" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd" is_pre_election: true
I20260809 22:44:21.642993 29991 raft_consensus.cc:2454] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 0.
I20260809 22:44:21.643126 30134 raft_consensus.cc:2454] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 0.
I20260809 22:44:21.643479 30208 leader_election.cc:302] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d001893dbd84c908be23f54d4e7e581, 1e6eb880515d4ea08b50eb9cf18f19c2; no voters:
I20260809 22:44:21.643839 30437 raft_consensus.cc:2781] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:21.643973 30437 raft_consensus.cc:482] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:21.644088 30437 raft_consensus.cc:3037] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.646318 30437 raft_consensus.cc:504] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.646899 30437 leader_election.cc:288] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 election: Requested vote from peers 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783), 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473)
I20260809 22:44:21.647439 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a312e7b41d0c45788ebd85428064f74c" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d001893dbd84c908be23f54d4e7e581"
I20260809 22:44:21.647430 30134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a312e7b41d0c45788ebd85428064f74c" candidate_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360667bc032e486590d876fd7e97afbd"
I20260809 22:44:21.647681 30134 raft_consensus.cc:3037] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.647698 29991 raft_consensus.cc:3037] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:21.650717 30134 raft_consensus.cc:2454] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 1.
I20260809 22:44:21.650732 29991 raft_consensus.cc:2454] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e6eb880515d4ea08b50eb9cf18f19c2 in term 1.
I20260809 22:44:21.651216 30210 leader_election.cc:302] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6eb880515d4ea08b50eb9cf18f19c2, 360667bc032e486590d876fd7e97afbd; no voters:
I20260809 22:44:21.651604 30437 raft_consensus.cc:2781] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:21.652043 30437 raft_consensus.cc:686] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1e6eb880515d4ea08b50eb9cf18f19c2, State: Running, Role: LEADER
I20260809 22:44:21.652386 30437 consensus_queue.cc:237] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } }
I20260809 22:44:21.655041 29814 catalog_manager.cc:5495] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e6eb880515d4ea08b50eb9cf18f19c2" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35237 } health_report { overall_health: HEALTHY } } }
I20260809 22:44:21.694979 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:21.689808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.21.27.131:35237)
0809 22:44:21.690332 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.690337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.690355 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:21.690406 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.690410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.690473 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.690623 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:21.691052 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.691825 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.693209 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.693616 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.693763 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.694201 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:21.694314 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:21.694419 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:21.694574 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 22:44:21.694655 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":302,"threads_started":1}
I20260809 22:44:21.704207 30134 raft_consensus.cc:1261] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Refusing update from remote peer 1e6eb880515d4ea08b50eb9cf18f19c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:21.704617 29991 raft_consensus.cc:1261] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Refusing update from remote peer 1e6eb880515d4ea08b50eb9cf18f19c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:21.705211 30437 consensus_queue.cc:1035] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360667bc032e486590d876fd7e97afbd" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 22:44:21.706290 30437 consensus_queue.cc:1035] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d001893dbd84c908be23f54d4e7e581" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:21.718976 30471 mvcc.cc:205] Tried to move back new op lower bound from 7316748131135287296 to 7316748130930425856. Current Snapshot: MvccSnapshot[applied={T|T < 7316748131135287296}]
I20260809 22:44:21.765545 29815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:21.765759 29815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:21.768577 29815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 370cd7341aed47dcab6f12ac352b7c1e: Sending DeleteTablet for 3 replicas of tablet a312e7b41d0c45788ebd85428064f74c
I20260809 22:44:21.769363 29964 tablet_service.cc:1514] Processing DeleteTablet for tablet a312e7b41d0c45788ebd85428064f74c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:58140
I20260809 22:44:21.769484 30111 tablet_service.cc:1514] Processing DeleteTablet for tablet a312e7b41d0c45788ebd85428064f74c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:33460
I20260809 22:44:21.769654 30480 tablet_replica.cc:330] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: stopping tablet replica
I20260809 22:44:21.769613 30259 tablet_service.cc:1514] Processing DeleteTablet for tablet a312e7b41d0c45788ebd85428064f74c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:21 UTC) from {username='oryx', principal='oryx/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:36426
I20260809 22:44:21.769768 30482 tablet_replica.cc:330] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: stopping tablet replica
I20260809 22:44:21.769846 30480 raft_consensus.cc:2229] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:21.769992 30480 raft_consensus.cc:2258] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.769979 30482 raft_consensus.cc:2229] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:21.770183 30482 raft_consensus.cc:2258] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.770255 30479 tablet_replica.cc:330] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: stopping tablet replica
I20260809 22:44:21.770455 30479 raft_consensus.cc:2229] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:21.770764 30479 raft_consensus.cc:2258] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:21.771203 30480 ts_tablet_manager.cc:1905] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.771214 30482 ts_tablet_manager.cc:1905] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.771704 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 29893
I20260809 22:44:21.771878 30479 ts_tablet_manager.cc:1905] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:21.777985 30482 ts_tablet_manager.cc:1918] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:21.778156 30482 log.cc:1196] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/a312e7b41d0c45788ebd85428064f74c
I20260809 22:44:21.778316 30479 ts_tablet_manager.cc:1918] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:21.778473 30479 log.cc:1196] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/a312e7b41d0c45788ebd85428064f74c
I20260809 22:44:21.778715 30482 ts_tablet_manager.cc:1939] T a312e7b41d0c45788ebd85428064f74c P 360667bc032e486590d876fd7e97afbd: Deleting consensus metadata
I20260809 22:44:21.778888 30479 ts_tablet_manager.cc:1939] T a312e7b41d0c45788ebd85428064f74c P 1e6eb880515d4ea08b50eb9cf18f19c2: Deleting consensus metadata
I20260809 22:44:21.778944 30480 ts_tablet_manager.cc:1918] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:44:21.779888 29799 catalog_manager.cc:4841] TS 1e6eb880515d4ea08b50eb9cf18f19c2 (127.21.27.131:35237): tablet a312e7b41d0c45788ebd85428064f74c (table test-table [id=d9b43157aecd4df087ccd724f9f60f63]) successfully deleted
I20260809 22:44:21.779980 29802 catalog_manager.cc:4841] TS 360667bc032e486590d876fd7e97afbd (127.21.27.130:40473): tablet a312e7b41d0c45788ebd85428064f74c (table test-table [id=d9b43157aecd4df087ccd724f9f60f63]) successfully deleted
I20260809 22:44:21.779111 30480 log.cc:1196] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/a312e7b41d0c45788ebd85428064f74c
I20260809 22:44:21.782102 30480 ts_tablet_manager.cc:1939] T a312e7b41d0c45788ebd85428064f74c P 0d001893dbd84c908be23f54d4e7e581: Deleting consensus metadata
I20260809 22:44:21.783180 29799 catalog_manager.cc:4841] TS 0d001893dbd84c908be23f54d4e7e581 (127.21.27.129:39783): tablet a312e7b41d0c45788ebd85428064f74c (table test-table [id=d9b43157aecd4df087ccd724f9f60f63]) successfully deleted
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.129@KRBTEST.COM
I20260809 22:44:21.795668 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:21.779299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.21.27.129:39783)
0809 22:44:21.781973 (+ 2674us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.781978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.782429 (+ 451us) server_negotiation.cc:407] Connection header received
0809 22:44:21.782590 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.782595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.782672 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.782799 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:21.784095 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.784807 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.785966 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.786344 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.786432 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.790415 (+ 3983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:21.790429 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:21.790446 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:21.790474 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:21.793189 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.793779 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.793786 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.793788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.793909 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.794230 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.794233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.794235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.794577 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:21.794816 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:21.795044 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:44:21.795300 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2546,"thread_start_us":2966,"threads_started":1}
I20260809 22:44:21.796326 30034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:21.922392 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 29893
I20260809 22:44:21.949061 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30043
Aug 09 22:44:21 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.130@KRBTEST.COM
I20260809 22:44:21.964445 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:21.949856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.21.27.130:40473)
0809 22:44:21.950035 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:21.950040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:21.950059 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:44:21.950149 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:21.950152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:21.950216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:21.950313 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:21.951614 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.952351 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.953572 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:21.953972 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:21.954075 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:21.958370 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:21.958384 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:21.958400 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:21.958437 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:21.961454 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.962170 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.962183 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.962186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.962316 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:21.962710 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:21.962717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:21.962720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:21.963091 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:21.963455 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:21.963886 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 22:44:21.964119 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:44:21.965104 30180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:22.090601 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30043
I20260809 22:44:22.116031 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30189
Aug 09 22:44:22 dist-test-slave-chsk krb5kdc[29765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.27.131@KRBTEST.COM
I20260809 22:44:22.131249 30490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:22.116843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.21.27.131:35237)
0809 22:44:22.117063 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:22.117068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:22.117192 (+ 124us) server_negotiation.cc:407] Connection header received
0809 22:44:22.117345 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:22.117349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:22.117424 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:22.117508 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:22.118781 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:22.119558 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:22.120795 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:22.121152 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:22.121250 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:22.125854 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:22.125871 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:22.125887 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:22.125920 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:22.128879 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:22.129471 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:22.129477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:22.129480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:22.129606 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:22.129910 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:22.129913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:22.129915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:22.130273 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:22.130493 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:22.130700 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 22:44:22.130865 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:44:22.132059 30326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:22.257035 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30189
I20260809 22:44:22.281641 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 29779
I20260809 22:44:22.282876 29861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:22.402308 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 29779
2026-08-09T22:44:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7287 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:22 dist-test-slave-chsk krb5kdc[30507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:22 dist-test-slave-chsk krb5kdc[30507](info): set up 2 sockets
Aug 09 22:44:22 dist-test-slave-chsk krb5kdc[30507](info): commencing operation
krb5kdc: starting...
W20260809 22:44:23.572306 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:23 dist-test-slave-chsk krb5kdc[30507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:44:23.832738 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37699
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:24.158981 30522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:24.159392 30522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:24.159556 30522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:24.166683 30522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:24.166810 30522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:24.166941 30522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:24.167039 30522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:24.167129 30522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:24.176774 30522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:24.177995 30522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:24.179467 30522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:24.191722 30527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:24.191911 30522 server_base.cc:1034] running on GCE node
W20260809 22:44:24.191759 30528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:24.191669 30530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:24.193040 30522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:24.194352 30522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:24.195622 30522 hybrid_clock.cc:648] HybridClock initialized: now 1786315464195577 us; error 48 us; skew 500 ppm
I20260809 22:44:24.195997 30522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T22:44:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1857 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:24 dist-test-slave-chsk krb5kdc[30536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:24 dist-test-slave-chsk krb5kdc[30536](info): set up 2 sockets
Aug 09 22:44:24 dist-test-slave-chsk krb5kdc[30536](info): commencing operation
krb5kdc: starting...
W20260809 22:44:25.422263 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:25 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:44:25.686014 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35739
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:26.022521 30551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:26.022840 30551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:26.022935 30551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:26.030977 30551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:26.031121 30551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:26.031370 30551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:26.031468 30551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:26.031517 30551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:26.042178 30551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:35739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:26.043491 30551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:26.044746 30551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:26.057224 30557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:26.057192 30558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:26.057129 30560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:26.060663 30551 server_base.cc:1034] running on GCE node
I20260809 22:44:26.061565 30551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:26.063267 30551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:26.064512 30551 hybrid_clock.cc:648] HybridClock initialized: now 1786315466064464 us; error 48 us; skew 500 ppm
I20260809 22:44:26.064882 30551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:26 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:26.073361 30551 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:44:26.075356 30551 webserver.cc:460] Webserver started at http://127.21.27.190:37873/ using document root <none> and password file <none>
I20260809 22:44:26.076117 30551 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:26.076227 30551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:26.076511 30551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:26.079782 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "75b973b77e1e486ba1301dd81e7a287d"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "826bd44c749eff3f8931c5440913c392"
server_key_iv: "abf8d0713ebce2681f0159649101b25b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.080556 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "75b973b77e1e486ba1301dd81e7a287d"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "826bd44c749eff3f8931c5440913c392"
server_key_iv: "abf8d0713ebce2681f0159649101b25b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.086141 30551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:44:26.089905 30567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:26.091441 30551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:26.091629 30551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "75b973b77e1e486ba1301dd81e7a287d"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "826bd44c749eff3f8931c5440913c392"
server_key_iv: "abf8d0713ebce2681f0159649101b25b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.091840 30551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:26.107400 30551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:26.111986 30551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:26.112303 30551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:26.132653 30551 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:35739
I20260809 22:44:26.132650 30624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:35739 every 8 connection(s)
I20260809 22:44:26.134230 30551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:26.138450 30625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:26.143107 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30551
I20260809 22:44:26.143428 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:26.144156 21614 external_mini_cluster.cc:1428] Setting key a841fe665eb4d515a31bef6e2339e9b8
I20260809 22:44:26.150462 30625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Bootstrap starting.
I20260809 22:44:26.153751 30625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:26.154827 30625 log.cc:824] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:26.157975 30625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: No bootstrap required, opened a new log
Aug 09 22:44:26 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:26.166903 30625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:26.167409 30625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:26.167587 30625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b973b77e1e486ba1301dd81e7a287d, State: Initialized, Role: FOLLOWER
I20260809 22:44:26.168185 30625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:26.168440 30625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:26.168591 30625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:26.168793 30625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:26.168812 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:26.146324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.21.27.190:35739)
0809 22:44:26.147368 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:26.147385 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:26.147432 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:44:26.149321 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:26.149348 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:26.149758 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:26.150189 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:26.151921 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.153237 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.154635 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.155562 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.155811 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:26.160267 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:26.160279 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:26.160298 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:26.160341 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:26.163821 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:26.164721 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:26.164729 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:26.164735 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:26.164924 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:26.165434 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:26.165439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:26.165443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:26.166108 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:26.166452 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:26.166915 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 22:44:26.167440 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":457,"threads_started":1}
I20260809 22:44:26.171836 30625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:26.172417 30625 leader_election.cc:302] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b973b77e1e486ba1301dd81e7a287d; no voters:
I20260809 22:44:26.172995 30625 leader_election.cc:288] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:26.173152 30630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:26.173926 30630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 LEADER]: Becoming Leader. State: Replica: 75b973b77e1e486ba1301dd81e7a287d, State: Running, Role: LEADER
I20260809 22:44:26.174469 30630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:26.175066 30625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:26.177613 30631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b973b77e1e486ba1301dd81e7a287d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } } }
I20260809 22:44:26.178119 30631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:26.177708 30632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b973b77e1e486ba1301dd81e7a287d. Latest consensus state: current_term: 1 leader_uuid: "75b973b77e1e486ba1301dd81e7a287d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } } }
I20260809 22:44:26.178607 30632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:26.180362 30637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:26.188395 30637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:26.198940 30637 catalog_manager.cc:1347] Generated new cluster ID: 73611142a8d84452b35c04bcd909b2d5
I20260809 22:44:26.199088 30637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:26.214134 30637 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:26.215535 30637 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:26.225310 30637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Generated new TSK 0
I20260809 22:44:26.226121 30637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:44:26.292714 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:26.623929 30652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:26.624215 30652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:26.624367 30652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:26.631362 30652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:26.631455 30652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:26.631625 30652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:26.640714 30652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:26.641901 30652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:26.643158 30652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:26.655815 30657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:26.655767 30660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:26.655854 30658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:26.656414 30652 server_base.cc:1034] running on GCE node
I20260809 22:44:26.657131 30652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:26.657940 30652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:26.659173 30652 hybrid_clock.cc:648] HybridClock initialized: now 1786315466659098 us; error 31 us; skew 500 ppm
I20260809 22:44:26.659488 30652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:26 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:26.662556 30652 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:44:26.663932 30652 webserver.cc:460] Webserver started at http://127.21.27.129:46109/ using document root <none> and password file <none>
I20260809 22:44:26.664551 30652 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:26.664680 30652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:26.664978 30652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:26.667980 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "eec5702390d444b69b8b6874588923bd"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "3ce1dfb92e956a94a46b17f8b1254e81"
server_key_iv: "f2f23ce5ec679b168a9517638cde895b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.668973 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "eec5702390d444b69b8b6874588923bd"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "3ce1dfb92e956a94a46b17f8b1254e81"
server_key_iv: "f2f23ce5ec679b168a9517638cde895b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.673710 30652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:44:26.677107 30668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:26.678287 30652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:26.678467 30652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "eec5702390d444b69b8b6874588923bd"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "3ce1dfb92e956a94a46b17f8b1254e81"
server_key_iv: "f2f23ce5ec679b168a9517638cde895b"
server_key_version: "encryptionkey@0"
I20260809 22:44:26.678646 30652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:26.693372 30652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:26.697278 30652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:26.697551 30652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:26.698432 30652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:26.699651 30652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:26.699731 30652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:26.699883 30652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:26.699968 30652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:26.733955 30652 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:41529
I20260809 22:44:26.734035 30790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:41529 every 8 connection(s)
I20260809 22:44:26.735684 30652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:26.738193 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30652
I20260809 22:44:26.738408 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:26.739348 21614 external_mini_cluster.cc:1428] Setting key 16cbf59304bf40be8e413dd29b0f64ab
Aug 09 22:44:26 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315466, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:26.758598 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:26.739201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:32973 (local address 127.21.27.190:35739)
0809 22:44:26.739745 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:26.739753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:26.741340 (+ 1587us) server_negotiation.cc:407] Connection header received
0809 22:44:26.744231 (+ 2891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:26.744240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:26.744347 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:26.744470 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:26.745993 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.747232 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.748716 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.749464 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.749614 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:26.752905 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:26.752910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:26.752913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:26.752942 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:26.755527 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:26.756284 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:26.756291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:26.756294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:26.756426 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:26.756886 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:26.756892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:26.756895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:26.757197 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:26.757355 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:26.758005 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 22:44:26.758231 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":291,"threads_started":1}
I20260809 22:44:26.759588 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:26.739647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:32973)
0809 22:44:26.741145 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:26.741191 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:26.743846 (+ 2655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:26.744810 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:26.744817 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:26.745302 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:26.745820 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:26.745837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.747363 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.747366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:26.748558 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:26.748580 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:26.749667 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:26.749671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:26.749864 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:26.750529 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:26.750549 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:26.752718 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:26.755709 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:26.755717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:26.755730 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:26.756161 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:26.756561 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:26.756566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:26.756570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:26.756760 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:26.757329 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:26.757336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:26.757645 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 22:44:26.757901 (+ 256us) client_negotiation.cc:240] Negotiation successful
0809 22:44:26.758389 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":988,"threads_started":1}
I20260809 22:44:26.761193 30791 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35739
I20260809 22:44:26.761483 30791 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:26.762172 30791 heartbeater.cc:511] Master 127.21.27.190:35739 requested a full tablet report, sending...
I20260809 22:44:26.764544 30586 ts_manager.cc:194] Registered new tserver with Master: eec5702390d444b69b8b6874588923bd (127.21.27.129:41529)
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
I20260809 22:44:26.766739 30586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:32973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:44:26.797885 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:27.125288 30797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:27.125561 30797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:27.125691 30797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:27.132822 30797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:27.132921 30797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:27.133095 30797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:27.142762 30797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:27.144014 30797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:27.145272 30797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:27.158489 30803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:27.158514 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:27.158612 30804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:27.159341 30797 server_base.cc:1034] running on GCE node
I20260809 22:44:27.159984 30797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:27.160882 30797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:27.162124 30797 hybrid_clock.cc:648] HybridClock initialized: now 1786315467162068 us; error 57 us; skew 500 ppm
I20260809 22:44:27.162436 30797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:27 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:27.165580 30797 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:44:27.167094 30797 webserver.cc:460] Webserver started at http://127.21.27.130:45289/ using document root <none> and password file <none>
I20260809 22:44:27.167922 30797 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:27.168047 30797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:27.168370 30797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:27.171357 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "5a1250c8c7924b9eb3d1aa7a24edef33"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "d446b215328e1b064584f7892c2bce5d"
server_key_iv: "98a1cbb1dec6ba7f8b7fc0015ae84159"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.172224 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "5a1250c8c7924b9eb3d1aa7a24edef33"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "d446b215328e1b064584f7892c2bce5d"
server_key_iv: "98a1cbb1dec6ba7f8b7fc0015ae84159"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.177459 30797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:44:27.180930 30814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.182390 30797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:44:27.182547 30797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "5a1250c8c7924b9eb3d1aa7a24edef33"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "d446b215328e1b064584f7892c2bce5d"
server_key_iv: "98a1cbb1dec6ba7f8b7fc0015ae84159"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.182741 30797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:27.203295 30797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:27.207075 30797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:27.207383 30797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:27.208290 30797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:27.209668 30797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:27.209745 30797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.209887 30797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:27.209942 30797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.243639 30797 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:36849
I20260809 22:44:27.243726 30938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:36849 every 8 connection(s)
I20260809 22:44:27.244901 30797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:27.253322 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30797
I20260809 22:44:27.253531 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:27.254096 21614 external_mini_cluster.cc:1428] Setting key fe6c983f18a4312c6faedda30601e477
Aug 09 22:44:27 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:27.265909 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:27.247435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:51557 (local address 127.21.27.190:35739)
0809 22:44:27.247609 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:27.247613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:27.248947 (+ 1334us) server_negotiation.cc:407] Connection header received
0809 22:44:27.250793 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:27.250797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:27.250869 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:27.250955 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:27.252426 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.253154 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.255338 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.255940 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.256109 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:27.260189 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:27.260195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:27.260198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:27.260228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:27.262870 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.263668 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.263672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.263675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.263795 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.264242 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.264246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.264248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.264496 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:27.264659 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:27.265329 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 22:44:27.265549 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:44:27.266829 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:27.248004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:51557)
0809 22:44:27.248786 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:27.248823 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:27.250461 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:27.251218 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:27.251226 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:27.251720 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:27.252273 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:27.252290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.253324 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.253330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:27.255195 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:27.255212 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.256177 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.256183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:27.256455 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:27.257211 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:27.257250 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:27.259937 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:27.263013 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:27.263022 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:27.263036 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:27.263528 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:27.263909 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:27.263914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:27.263918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:27.264113 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:27.264634 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:27.264642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:27.264971 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 22:44:27.265225 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 22:44:27.265723 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":338,"threads_started":1}
I20260809 22:44:27.268356 30939 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35739
I20260809 22:44:27.268653 30939 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:27.269321 30939 heartbeater.cc:511] Master 127.21.27.190:35739 requested a full tablet report, sending...
I20260809 22:44:27.270826 30586 ts_manager.cc:194] Registered new tserver with Master: 5a1250c8c7924b9eb3d1aa7a24edef33 (127.21.27.130:36849)
I20260809 22:44:27.272133 30586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:51557
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:44:27.310631 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:27.640965 30944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:27.641252 30944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:27.641348 30944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:27.648619 30944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:27.648726 30944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:27.648835 30944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:27.658730 30944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:27.659947 30944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:27.661190 30944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:27.673977 30950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:27.673931 30951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:27.674093 30953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:27.674819 30944 server_base.cc:1034] running on GCE node
I20260809 22:44:27.675544 30944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:27.676555 30944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:27.677783 30944 hybrid_clock.cc:648] HybridClock initialized: now 1786315467677738 us; error 59 us; skew 500 ppm
I20260809 22:44:27.678098 30944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:27 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:27.681805 30944 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:44:27.683415 30944 webserver.cc:460] Webserver started at http://127.21.27.131:46607/ using document root <none> and password file <none>
I20260809 22:44:27.684163 30944 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:27.684311 30944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:27.684630 30944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:27.687954 30944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "93e344f291aa43789cabf50c64174c96"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "c3a3a62a93ebb1f7044a8c43449d64f6"
server_key_iv: "87a4c079b659edda8d83326589ff5bc0"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.688757 30944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "93e344f291aa43789cabf50c64174c96"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "c3a3a62a93ebb1f7044a8c43449d64f6"
server_key_iv: "87a4c079b659edda8d83326589ff5bc0"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.694015 30944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:44:27.697594 30962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.698841 30944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:27.699045 30944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "93e344f291aa43789cabf50c64174c96"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "c3a3a62a93ebb1f7044a8c43449d64f6"
server_key_iv: "87a4c079b659edda8d83326589ff5bc0"
server_key_version: "encryptionkey@0"
I20260809 22:44:27.699316 30944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:27.722666 30944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:27.727212 30944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:27.727512 30944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:27.728645 30944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:27.732370 30944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:27.732491 30944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.732625 30944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:27.732687 30944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:27.769244 30944 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45371
I20260809 22:44:27.769308 31090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45371 every 8 connection(s)
I20260809 22:44:27.769727 30791 heartbeater.cc:503] Master 127.21.27.190:35739 was elected leader, sending a full tablet report...
I20260809 22:44:27.770838 30944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:27.775723 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 30944
I20260809 22:44:27.775903 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:27.776367 21614 external_mini_cluster.cc:1428] Setting key e9898c00b9c19bdd2e60a6696eb74edc
Aug 09 22:44:27 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315467, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:27.791467 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:27.773538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:37863 (local address 127.21.27.190:35739)
0809 22:44:27.773985 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:27.773989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:27.774943 (+ 954us) server_negotiation.cc:407] Connection header received
0809 22:44:27.777348 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:27.777354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:27.777445 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:27.777575 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:27.779044 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.779914 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.781560 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.782080 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.782215 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:27.785673 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:27.785680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:27.785684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:27.785725 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:27.788389 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.789147 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.789152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.789156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.789286 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.789717 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.789722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.789725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.789998 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:27.790156 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:27.790826 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 22:44:27.791046 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":221,"threads_started":1}
I20260809 22:44:27.792375 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:27.774115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:37863)
0809 22:44:27.774796 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:27.774833 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:27.776989 (+ 2156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:27.777816 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:27.777823 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:27.778332 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:27.778872 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:27.778893 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.780074 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.780080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:27.781418 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:27.781429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.782303 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.782311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:27.782560 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:27.783284 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:27.783306 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:27.785428 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:27.788526 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:27.788537 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:27.788554 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:27.788971 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:27.789398 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:27.789403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:27.789407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:27.789605 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:27.790137 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:27.790144 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:27.790458 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 22:44:27.790726 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 22:44:27.791271 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":266,"threads_started":1}
I20260809 22:44:27.793781 31091 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35739
I20260809 22:44:27.794054 31091 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:27.794714 31091 heartbeater.cc:511] Master 127.21.27.190:35739 requested a full tablet report, sending...
I20260809 22:44:27.796167 30586 ts_manager.cc:194] Registered new tserver with Master: 93e344f291aa43789cabf50c64174c96 (127.21.27.131:45371)
I20260809 22:44:27.797058 30586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:37863
I20260809 22:44:27.803817 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
Entry for principal oryx/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab.
WARNING: no policy specified for oryx/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
Entry for principal oryx/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab.
WARNING: no policy specified for oryx/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
Entry for principal oryx/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab.
I20260809 22:44:27.961366 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30652
Aug 09 22:44:27 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:44:27.977708 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:27.962553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.21.27.129:41529)
0809 22:44:27.963264 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:27.963271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:27.963302 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:44:27.963438 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:27.963446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:27.963657 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:27.963806 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:27.965152 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.966169 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.967471 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:27.968063 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:27.968164 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:27.972184 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:27.972195 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:27.972209 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:27.972248 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:27.975214 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.975880 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.975891 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.975897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.976035 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:27.976393 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:27.976397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:27.976399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:27.976730 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:27.976949 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:27.977218 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 22:44:27.977417 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":319,"threads_started":1}
I20260809 22:44:27.978938 30786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:28.062553 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30652
I20260809 22:44:28.081755 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30797
Aug 09 22:44:28 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:44:28.098194 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:28.082742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38838 (local address 127.21.27.130:36849)
0809 22:44:28.083313 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:28.083320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:28.083346 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:44:28.083407 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:28.083413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:28.083620 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:28.083771 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:28.085063 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:28.086107 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:28.087636 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:28.088203 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:28.088312 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:28.092566 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:28.092577 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:28.092590 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:28.092624 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:28.095571 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:28.096296 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:28.096307 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:28.096314 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:28.096437 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:28.096767 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:28.096771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:28.096773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:28.097104 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:28.097346 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:28.097577 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 22:44:28.097773 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":243,"threads_started":1}
I20260809 22:44:28.099521 30934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:28.183842 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30797
I20260809 22:44:28.202419 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30944
Aug 09 22:44:28 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:44:28.220263 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:28.203364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.21.27.131:45371)
0809 22:44:28.203899 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:28.203906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:28.203936 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:44:28.204003 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:28.204009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:28.204223 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:28.204406 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:28.205802 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:28.206840 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:28.208265 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:28.208831 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:28.208980 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:28.213861 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:28.213877 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:28.213893 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:28.213935 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:28.217129 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:28.217961 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:28.217968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:28.217971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:28.218107 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:28.218524 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:28.218528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:28.218530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:28.218897 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:28.219253 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:28.219606 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:44:28.219901 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260809 22:44:28.221570 31086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:28.307060 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30944
I20260809 22:44:28.327090 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 30551
I20260809 22:44:28.328579 30620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:28.437098 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 30551
I20260809 22:44:28.460057 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35739
--webserver_interface=127.21.27.190
--webserver_port=37873
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:28.811326 31112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:28.811673 31112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:28.811897 31112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:28.819820 31112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:28.819957 31112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:28.820063 31112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:28.820183 31112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:28.820237 31112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:28.830742 31112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:35739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:28.832245 31112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:28.833657 31112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:28.846576 31117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:28.846606 31118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:28.846741 31120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:28.847368 31112 server_base.cc:1034] running on GCE node
I20260809 22:44:28.848227 31112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:28.849880 31112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:28.851321 31112 hybrid_clock.cc:648] HybridClock initialized: now 1786315468851107 us; error 63 us; skew 500 ppm
I20260809 22:44:28.851714 31112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:28 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315468, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:28.855016 31112 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:44:28.856770 31112 webserver.cc:460] Webserver started at http://127.21.27.190:37873/ using document root <none> and password file <none>
I20260809 22:44:28.857630 31112 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:28.857803 31112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:28.863813 31112 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:44:28.866936 31127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:28.868713 31112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:44:28.868937 31112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "75b973b77e1e486ba1301dd81e7a287d"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "826bd44c749eff3f8931c5440913c392"
server_key_iv: "abf8d0713ebce2681f0159649101b25b"
server_key_version: "encryptionkey@0"
I20260809 22:44:28.870028 31112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:28.899420 31112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:28.904043 31112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:28.904374 31112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:28.925366 31112 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:35739
I20260809 22:44:28.925410 31184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:35739 every 8 connection(s)
I20260809 22:44:28.926990 31112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:28.929306 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31112
I20260809 22:44:28.930164 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:41529
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:35739
--webserver_port=46109
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:44:28.934468 31185 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:44:28.938141 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Bootstrap starting.
I20260809 22:44:28.957881 31185 log.cc:824] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:28.967203 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:44:28.967782 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Bootstrap complete.
I20260809 22:44:28.977421 31185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:28.978328 31185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b973b77e1e486ba1301dd81e7a287d, State: Initialized, Role: FOLLOWER
I20260809 22:44:28.978945 31185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:28.979229 31185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:28.979368 31185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:28.979571 31185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:44:28.983520 31185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:28.984032 31185 leader_election.cc:302] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b973b77e1e486ba1301dd81e7a287d; no voters:
I20260809 22:44:28.984714 31185 leader_election.cc:288] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:44:28.984851 31189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:44:28.986315 31189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [term 2 LEADER]: Becoming Leader. State: Replica: 75b973b77e1e486ba1301dd81e7a287d, State: Running, Role: LEADER
I20260809 22:44:28.986910 31189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } }
I20260809 22:44:28.987198 31185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:28.989676 31190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75b973b77e1e486ba1301dd81e7a287d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } } }
I20260809 22:44:28.990144 31190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:28.994050 31191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b973b77e1e486ba1301dd81e7a287d. Latest consensus state: current_term: 2 leader_uuid: "75b973b77e1e486ba1301dd81e7a287d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b973b77e1e486ba1301dd81e7a287d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35739 } } }
I20260809 22:44:28.994469 31191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:28.996320 31195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:29.003676 31195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:29.005687 31195 catalog_manager.cc:1259] Loaded cluster ID: 73611142a8d84452b35c04bcd909b2d5
I20260809 22:44:29.005836 31195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:29.011445 31195 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:29.013535 31195 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75b973b77e1e486ba1301dd81e7a287d: Loaded TSK: 0
I20260809 22:44:29.015585 31195 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:44:29.307567 31187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:29.307906 31187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:29.308084 31187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:29.315641 31187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:29.315801 31187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:29.315980 31187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:29.326283 31187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.129.keytab
--principal=oryx/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:41529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=46109
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:29.327828 31187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:29.329172 31187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:29.344337 31215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:29.344309 31214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:29.345099 31217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:29.345400 31187 server_base.cc:1034] running on GCE node
I20260809 22:44:29.346058 31187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:29.347211 31187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:29.348464 31187 hybrid_clock.cc:648] HybridClock initialized: now 1786315469348397 us; error 54 us; skew 500 ppm
I20260809 22:44:29.348834 31187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315469, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:29.356051 31187 init.cc:377] Logged in from keytab as oryx/127.21.27.129@KRBTEST.COM (short username oryx)
I20260809 22:44:29.357976 31187 webserver.cc:460] Webserver started at http://127.21.27.129:46109/ using document root <none> and password file <none>
I20260809 22:44:29.358848 31187 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:29.358999 31187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:29.366324 31187 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:44:29.369797 31226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.371628 31187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:29.371822 31187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "eec5702390d444b69b8b6874588923bd"
format_stamp: "Formatted at 2026-08-09 22:44:26 on dist-test-slave-chsk"
server_key: "3ce1dfb92e956a94a46b17f8b1254e81"
server_key_iv: "f2f23ce5ec679b168a9517638cde895b"
server_key_version: "encryptionkey@0"
I20260809 22:44:29.373199 31187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:29.398240 31187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:29.403903 31187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:29.404300 31187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:29.405370 31187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:29.406695 31187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:29.406802 31187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.406886 31187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:29.406942 31187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.441520 31187 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:41529
I20260809 22:44:29.441579 31353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:41529 every 8 connection(s)
I20260809 22:44:29.443037 31187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:29.449285 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31187
I20260809 22:44:29.449985 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:36849
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:35739
--webserver_port=45289
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:29.821661 31358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:29.821978 31358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:29.822160 31358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:29.829766 31358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:29.829903 31358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:29.830109 31358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:29.840453 31358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.130.keytab
--principal=oryx/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:36849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=45289
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:29.841869 31358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:29.843398 31358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:29.862021 31365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:29.862130 31367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:29.862004 31364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:29.864898 31358 server_base.cc:1034] running on GCE node
I20260809 22:44:29.865823 31358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:29.867192 31358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:29.868415 31358 hybrid_clock.cc:648] HybridClock initialized: now 1786315469868358 us; error 58 us; skew 500 ppm
I20260809 22:44:29.868762 31358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315469, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:29.872813 31358 init.cc:377] Logged in from keytab as oryx/127.21.27.130@KRBTEST.COM (short username oryx)
I20260809 22:44:29.874652 31358 webserver.cc:460] Webserver started at http://127.21.27.130:45289/ using document root <none> and password file <none>
I20260809 22:44:29.875489 31358 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:29.875655 31358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:29.881790 31358 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 22:44:29.884828 31375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.886276 31358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:29.886456 31358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "5a1250c8c7924b9eb3d1aa7a24edef33"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "d446b215328e1b064584f7892c2bce5d"
server_key_iv: "98a1cbb1dec6ba7f8b7fc0015ae84159"
server_key_version: "encryptionkey@0"
I20260809 22:44:29.887776 31358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:29.910979 31358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:29.915597 31358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:29.915874 31358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:29.916978 31358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:29.918527 31358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:29.918654 31358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.918776 31358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:29.918866 31358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:29.966980 31503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:36849 every 8 connection(s)
I20260809 22:44:29.966993 31358 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:36849
I20260809 22:44:29.969779 31358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:29.979847 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31358
I20260809 22:44:29.980927 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:45371
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:35739
--webserver_port=46607
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:29 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:30.324916 31508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:30.325230 31508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:30.325410 31508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:30.332741 31508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:30.332883 31508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:30.333072 31508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:30.342839 31508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/krb5kdc/oryx_127.21.27.131.keytab
--principal=oryx/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:45371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=46607
--tserver_master_addrs=127.21.27.190:35739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:30.344213 31508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:30.345507 31508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:30.363101 31514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:30.371441 31515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:30.372141 31508 server_base.cc:1034] running on GCE node
W20260809 22:44:30.373641 31517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:30.375118 31508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:30.376982 31508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:30.378484 31508 hybrid_clock.cc:648] HybridClock initialized: now 1786315470378402 us; error 63 us; skew 500 ppm
I20260809 22:44:30.379189 31508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315470, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:30.385846 31508 init.cc:377] Logged in from keytab as oryx/127.21.27.131@KRBTEST.COM (short username oryx)
I20260809 22:44:30.388993 31508 webserver.cc:460] Webserver started at http://127.21.27.131:46607/ using document root <none> and password file <none>
I20260809 22:44:30.390461 31508 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:30.390781 31508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:30.401732 31508 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 22:44:30.408121 31525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:30.410493 31508 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:44:30.410808 31508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "93e344f291aa43789cabf50c64174c96"
format_stamp: "Formatted at 2026-08-09 22:44:27 on dist-test-slave-chsk"
server_key: "c3a3a62a93ebb1f7044a8c43449d64f6"
server_key_iv: "87a4c079b659edda8d83326589ff5bc0"
server_key_version: "encryptionkey@0"
I20260809 22:44:30.413030 31508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:30.481616 31508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:30.490331 31508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:30.490869 31508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:30.492825 31508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:30.495555 31508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:30.495788 31508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:30.496021 31508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:30.496381 31508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:30.560105 31508 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45371
I20260809 22:44:30.560235 31649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45371 every 8 connection(s)
I20260809 22:44:30.563024 31508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:30.567807 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31508
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:30.659574 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:30.570412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.21.27.190:35739)
0809 22:44:30.617689 (+ 47277us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:30.617697 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:30.623366 (+ 5669us) server_negotiation.cc:407] Connection header received
0809 22:44:30.623463 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:30.623473 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:30.623596 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:30.623743 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:30.631383 (+ 7640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.633340 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.637657 (+ 4317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.638707 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.638858 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.644529 (+ 5671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:30.644546 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:30.644579 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:30.644666 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:30.651353 (+ 6687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:30.652743 (+ 1390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:30.652758 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:30.652774 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:30.653103 (+ 329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:30.653758 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:30.653766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:30.653772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:30.654691 (+ 919us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:30.655176 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:30.657186 (+ 2010us) server_negotiation.cc:299] Negotiation successful
0809 22:44:30.657873 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47132,"thread_start_us":51358,"threads_started":1}
W20260809 22:44:30.917739 31357 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.461s user 0.000s sys 0.014s
W20260809 22:44:30.920135 31356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:29.446094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36735 (local address 127.21.27.190:35739)
0809 22:44:29.446980 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:29.446997 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:29.447269 (+ 272us) server_negotiation.cc:407] Connection header received
0809 22:44:29.449676 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:29.449687 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:29.449992 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:29.450435 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:29.451988 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.453215 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.454731 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.455450 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.455645 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.919630 (+1463985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:36735: BlockingRecv error: recv got EOF from 127.21.27.129:36735 (error 108)
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":340,"threads_started":1}
I20260809 22:44:30.922688 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:29.446162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:36735)
0809 22:44:29.447025 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:29.447095 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:29.449020 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:29.450572 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:29.450584 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:29.451271 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:29.451793 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:29.451811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.453390 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.453394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:29.454573 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:29.454585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.455718 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.455726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:29.455977 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:29.456674 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:29.456710 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:30.919125 (+1462415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":354,"threads_started":1}
W20260809 22:44:30.924793 31354 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:35739 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.27.190:35739: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:30.960705 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:30.927673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:45591)
0809 22:44:30.929784 (+ 2111us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:30.929815 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:30.929982 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:30.939313 (+ 9331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:30.939322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:30.940090 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:30.940453 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.940472 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.942768 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.942777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.945231 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.945252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.947251 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.947261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.947542 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.948466 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:30.948501 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:30.960208 (+ 11707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1925,"thread_start_us":29540,"threads_started":1}
W20260809 22:44:30.960951 31356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:30.931337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:45591 (local address 127.21.27.190:35739)
0809 22:44:30.931591 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:30.931601 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:30.931632 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:44:30.931715 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:30.931723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:30.931833 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:30.931983 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:30.940727 (+ 8744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.942465 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.945493 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.946504 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.946638 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.960577 (+ 13939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:45591: BlockingRecv error: recv got EOF from 127.21.27.129:45591 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:30 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:30.983132 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:30.961730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:57845)
0809 22:44:30.962131 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:30.962157 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:30.962304 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:30.963187 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:30.963195 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:30.963873 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:30.964206 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.964224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.966780 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.966790 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.969242 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.969264 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.970955 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.970965 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.971290 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.972097 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:30.972129 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:30.982662 (+ 10533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229}
W20260809 22:44:30.983500 31356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:30.962081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:57845 (local address 127.21.27.190:35739)
0809 22:44:30.962717 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:30.962726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:30.962752 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:44:30.962827 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:30.962836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:30.962939 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:30.963097 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:30.964372 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.966173 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.969487 (+ 3314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.970462 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.970595 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.983040 (+ 12445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:57845: BlockingRecv error: recv got EOF from 127.21.27.129:57845 (error 108)
Metrics: {"server-negotiator.queue_time_us":467}
W20260809 22:44:30.984087 31354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:44:31.312398 31507 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.330s user 0.004s sys 0.011s
W20260809 22:44:31.313750 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:29.972841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:56349 (local address 127.21.27.190:35739)
0809 22:44:29.973466 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:29.973473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:29.974391 (+ 918us) server_negotiation.cc:407] Connection header received
0809 22:44:29.976414 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:29.976420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:29.976529 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:29.976660 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:29.978142 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.979012 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.980724 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.981237 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.981362 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.313420 (+1332058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:56349: BlockingRecv error: recv got EOF from 127.21.27.130:56349 (error 108)
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":292,"threads_started":1}
I20260809 22:44:31.314312 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:29.973311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:56349)
0809 22:44:29.974228 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:29.974280 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:29.976148 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:29.976907 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:29.976915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:29.977431 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:29.977957 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:29.977974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.979209 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.979215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:29.980580 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:29.980593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:29.981631 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:29.981636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:29.981823 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:29.982520 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:29.982548 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.313213 (+1330665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":400,"threads_started":1}
W20260809 22:44:31.315361 31504 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:35739 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.27.190:35739: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:31.333549 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:31.316217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:60851)
0809 22:44:31.316461 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:31.316478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:31.316624 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:31.317030 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:31.317038 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:31.317745 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:31.318058 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.318088 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.319846 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.319855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.322283 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.322306 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.323297 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.323306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.323579 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.327719 (+ 4140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:31.327758 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.333229 (+ 5471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260809 22:44:31.333685 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:31.316179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:60851 (local address 127.21.27.190:35739)
0809 22:44:31.316369 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:31.316377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:31.316557 (+ 180us) server_negotiation.cc:407] Connection header received
0809 22:44:31.316763 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:31.316767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:31.316843 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:31.316942 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:31.318337 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.319090 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.322573 (+ 3483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.323055 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.323233 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.333434 (+ 10201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:60851: BlockingRecv error: recv got EOF from 127.21.27.130:60851 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:31.343730 31506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:31.334680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:52473 (local address 127.21.27.190:35739)
0809 22:44:31.334880 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:31.334884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:31.334897 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:44:31.334944 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:31.334948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:31.335010 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:31.335117 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:31.335790 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.336522 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.337762 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.338165 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.338298 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.343457 (+ 5159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:52473: BlockingRecv error: recv got EOF from 127.21.27.130:52473 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:44:31.344224 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:31.334414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:52473)
0809 22:44:31.334611 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:31.334626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:31.334704 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:31.335170 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:31.335174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:31.335501 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:31.335652 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.335660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.336672 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.336677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.337625 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.337637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.338309 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.338314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.338503 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.339088 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:31.339105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.343946 (+ 4841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":81}
W20260809 22:44:31.344548 31504 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:44:31.828837 31655 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.178s user 0.004s sys 0.022s
W20260809 22:44:31.830014 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:30.569911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:38951 (local address 127.21.27.190:35739)
0809 22:44:30.617234 (+ 47323us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:30.617247 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:30.622493 (+ 5246us) server_negotiation.cc:407] Connection header received
0809 22:44:30.637465 (+ 14972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:30.637476 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:30.637627 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:30.637831 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:30.641750 (+ 3919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.643637 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.647333 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.648361 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.648528 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.829814 (+1181286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:38951: BlockingRecv error: recv got EOF from 127.21.27.131:38951 (error 108)
Metrics: {"server-negotiator.queue_time_us":47074,"thread_start_us":51666,"threads_started":1}
I20260809 22:44:31.830574 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:30.572849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:38951)
0809 22:44:30.618687 (+ 45838us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:30.618793 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:30.630715 (+ 11922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:30.638769 (+ 8054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:30.638790 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:30.640110 (+ 1320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:30.641393 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.641431 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.643932 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.643940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.647025 (+ 3085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:30.647065 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:30.649109 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:30.649119 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:30.649570 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:30.650611 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:30.650657 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.829623 (+1178966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45280,"thread_start_us":48303,"threads_started":1}
W20260809 22:44:31.831493 31650 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:35739 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.27.190:35739: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:31.840756 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:31.832126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:36651)
0809 22:44:31.832320 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:31.832337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:31.832431 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:31.832715 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:31.832718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:31.833056 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:31.833239 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.833248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.834246 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.834249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.835281 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.835294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.836059 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.836064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.836197 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.836885 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:31.836903 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.840556 (+ 3653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 22:44:31.841003 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:31.832243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:36651 (local address 127.21.27.190:35739)
0809 22:44:31.832410 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:31.832414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:31.832432 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:44:31.832528 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:31.832531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:31.832590 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:31.832677 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:31.833373 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.834124 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.835411 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.835873 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.835973 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.840740 (+ 4767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:36651: BlockingRecv error: recv got EOF from 127.21.27.131:36651 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:31.849681 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:31.841459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:51563)
0809 22:44:31.841702 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:31.841724 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:31.841833 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:31.842163 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:31.842166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:31.842432 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:31.842579 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.842586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.843583 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.843587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.844557 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.844569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.845268 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.845273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.845422 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.845927 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:31.845943 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.849418 (+ 3475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143}
W20260809 22:44:31.849998 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:31.841630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:51563 (local address 127.21.27.190:35739)
0809 22:44:31.841877 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:31.841881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:31.841896 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:44:31.841950 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:31.841953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:31.842026 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:31.842111 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:31.842695 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.843467 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.844694 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.845127 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.845225 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.849721 (+ 4496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:51563: BlockingRecv error: recv got EOF from 127.21.27.131:51563 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
W20260809 22:44:31.850102 31650 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:31 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:31.994318 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:31.984685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:44651)
0809 22:44:31.985203 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:31.985219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:31.985334 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:31.985650 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:31.985653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:31.986081 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:31.986233 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.986242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.987276 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.987280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.988332 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:31.988345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.989168 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.989173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:31.989297 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.989887 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:31.989904 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:31.993958 (+ 4054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":301,"threads_started":1}
W20260809 22:44:31.994455 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:31.984757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:44651 (local address 127.21.27.190:35739)
0809 22:44:31.984922 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:31.984926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:31.985318 (+ 392us) server_negotiation.cc:407] Connection header received
0809 22:44:31.985422 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:31.985426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:31.985522 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:31.985613 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:31.986376 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.987108 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.988476 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:31.988973 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:31.989096 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:31.994193 (+ 5097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:44651: BlockingRecv error: recv got EOF from 127.21.27.129:44651 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:32.354849 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:32.345244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:43475)
0809 22:44:32.345877 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:32.345897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:32.346063 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:32.346528 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:32.346533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:32.347193 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:32.347396 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.347409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.348455 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.348458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.349425 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.349437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.350264 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.350269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.350401 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:32.351051 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:32.351070 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:32.354609 (+ 3539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":331,"threads_started":1}
W20260809 22:44:32.355144 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:32.345360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43475 (local address 127.21.27.190:35739)
0809 22:44:32.345589 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:32.345598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:32.345977 (+ 379us) server_negotiation.cc:407] Connection header received
0809 22:44:32.346268 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:32.346274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:32.346359 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:32.346453 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:32.347542 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.348323 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.349548 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.349964 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.350053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:32.354854 (+ 4801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:43475: BlockingRecv error: recv got EOF from 127.21.27.130:43475 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:32.859956 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:32.850841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:37481)
0809 22:44:32.851473 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:32.851493 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:32.851607 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:32.851912 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:32.851916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:32.852342 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:32.852488 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.852496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.853519 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.853523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.854484 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.854495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.855517 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.855523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.855692 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:32.856323 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:32.856341 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:32.859725 (+ 3384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":392,"threads_started":1}
W20260809 22:44:32.860193 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:32.850897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:37481 (local address 127.21.27.190:35739)
0809 22:44:32.851081 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:32.851085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:32.851569 (+ 484us) server_negotiation.cc:407] Connection header received
0809 22:44:32.851705 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:32.851708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:32.851782 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:32.851869 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:32.852612 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.853383 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.854627 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.855257 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.855402 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:32.859926 (+ 4524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:37481: BlockingRecv error: recv got EOF from 127.21.27.131:37481 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:32 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:33.004194 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:32.995446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:37353)
0809 22:44:32.995988 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:32.996004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:32.996160 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:32.996467 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:32.996469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:32.996874 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:32.997050 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.997058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.998104 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.998108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.999024 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:32.999034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.999691 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.999695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:32.999824 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.000495 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:33.000514 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:33.003945 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":286,"threads_started":1}
W20260809 22:44:33.004500 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:32.995485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:37353 (local address 127.21.27.190:35739)
0809 22:44:32.995664 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:32.995669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:32.996143 (+ 474us) server_negotiation.cc:407] Connection header received
0809 22:44:32.996266 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:32.996270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:32.996337 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:32.996424 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:32.997210 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.997986 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.999160 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:32.999555 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:32.999678 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.004179 (+ 4501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:37353: BlockingRecv error: recv got EOF from 127.21.27.129:37353 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:44:33 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:33 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:33.365048 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:33.355857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:45057)
0809 22:44:33.356413 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:33.356430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:33.356600 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:33.356951 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:33.356956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:33.357434 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:33.357634 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:33.357643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.358659 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.358665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:33.359626 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:33.359638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.360374 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.360381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:33.360553 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.361132 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:33.361149 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:33.364693 (+ 3544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":313,"threads_started":1}
W20260809 22:44:33.365124 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:33.355952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:45057 (local address 127.21.27.190:35739)
0809 22:44:33.356272 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:33.356278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:33.356523 (+ 245us) server_negotiation.cc:407] Connection header received
0809 22:44:33.356737 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:33.356744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:33.356813 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:33.356919 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:33.357786 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.358508 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.359773 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.360205 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.360336 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.364886 (+ 4550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:45057: BlockingRecv error: recv got EOF from 127.21.27.130:45057 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 09 22:44:33 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:33 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:33.870604 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:33.860899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:59299)
0809 22:44:33.861490 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:33.861517 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:33.861640 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:33.862027 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:33.862033 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:33.862471 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:33.862673 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:33.862684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.863945 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.863950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:33.865169 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:33.865190 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.866026 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.866031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:33.866164 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.866752 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:33.866770 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:33.870354 (+ 3584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":366,"threads_started":1}
W20260809 22:44:33.870793 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:33.861173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:59299 (local address 127.21.27.190:35739)
0809 22:44:33.861358 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:33.861363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:33.861626 (+ 263us) server_negotiation.cc:407] Connection header received
0809 22:44:33.861733 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:33.861737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:33.861828 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:33.861923 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:33.862828 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.863793 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.865325 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:33.865890 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:33.865997 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:33.870557 (+ 4560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:59299: BlockingRecv error: recv got EOF from 127.21.27.131:59299 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:34.014875 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:34.005244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:53071)
0809 22:44:34.005739 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:34.005755 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:34.005874 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:34.006183 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:34.006186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:34.006555 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:34.006725 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.006758 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.007766 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.007770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.008706 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.008716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.009861 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.009867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.010030 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.010769 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:34.010788 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:34.014633 (+ 3845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":249,"threads_started":1}
W20260809 22:44:34.015085 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:34.005297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:53071 (local address 127.21.27.190:35739)
0809 22:44:34.005535 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:34.005542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:34.005846 (+ 304us) server_negotiation.cc:407] Connection header received
0809 22:44:34.005978 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:34.005981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:34.006045 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:34.006128 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:34.006898 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.007654 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.008885 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.009667 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.009845 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.014864 (+ 5019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:53071: BlockingRecv error: recv got EOF from 127.21.27.129:53071 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:34.375134 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:34.366224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:54979)
0809 22:44:34.366835 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:34.366851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:34.367009 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:34.367466 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:34.367471 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:34.367864 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:34.368104 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.368115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.369118 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.369123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.370022 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.370033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.370714 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.370719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.370836 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.371461 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:34.371478 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:34.374796 (+ 3318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":321,"threads_started":1}
W20260809 22:44:34.375324 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:34.366336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:54979 (local address 127.21.27.190:35739)
0809 22:44:34.366566 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:34.366576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:34.366935 (+ 359us) server_negotiation.cc:407] Connection header received
0809 22:44:34.367214 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:34.367220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:34.367303 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:34.367420 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:34.368244 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.368986 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.370157 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.370579 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.370673 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.375030 (+ 4357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:54979: BlockingRecv error: recv got EOF from 127.21.27.130:54979 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:34 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:34.880765 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:34.871590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:40973)
0809 22:44:34.872062 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:34.872079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:34.872207 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:34.872515 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:34.872518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:34.872868 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:34.873033 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.873042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.874114 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.874120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.875302 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:34.875315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.876238 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.876246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:34.876425 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.877003 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:34.877020 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:34.880492 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":253,"threads_started":1}
W20260809 22:44:34.881085 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:34.871675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:40973 (local address 127.21.27.190:35739)
0809 22:44:34.871913 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:34.871921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:34.872162 (+ 241us) server_negotiation.cc:407] Connection header received
0809 22:44:34.872306 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:34.872310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:34.872388 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:34.872473 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:34.873159 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.873939 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.875454 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:34.876081 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:34.876180 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:34.880785 (+ 4605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:40973: BlockingRecv error: recv got EOF from 127.21.27.131:40973 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:35.025214 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:35.015952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:60943)
0809 22:44:35.016450 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:35.016476 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:35.016591 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:35.016909 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:35.016911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:35.017446 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:35.017636 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.017646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.018752 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.018758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.019785 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.019796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.020628 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.020634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.020808 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.021397 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:35.021414 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:35.024940 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":288,"threads_started":1}
W20260809 22:44:35.025425 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:35.016059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:60943 (local address 127.21.27.190:35739)
0809 22:44:35.016265 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:35.016269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:35.016553 (+ 284us) server_negotiation.cc:407] Connection header received
0809 22:44:35.016701 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:35.016704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:35.016766 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:35.016855 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:35.017807 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.018603 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.019941 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.020474 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.020606 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.025169 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:60943: BlockingRecv error: recv got EOF from 127.21.27.129:60943 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:35.385952 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:35.376139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:37175)
0809 22:44:35.376699 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:35.376715 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:35.376870 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:35.377243 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:35.377248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:35.377681 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:35.377856 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.377865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.379024 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.379030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.380175 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.380188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.380927 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.380932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.381121 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.381852 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:35.381871 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:35.385569 (+ 3698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":317,"threads_started":1}
W20260809 22:44:35.386237 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:35.376235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:37175 (local address 127.21.27.190:35739)
0809 22:44:35.376439 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:35.376446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:35.376802 (+ 356us) server_negotiation.cc:407] Connection header received
0809 22:44:35.377003 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:35.377008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:35.377099 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:35.377211 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:35.378001 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.378889 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.380328 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.380776 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.380913 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.385809 (+ 4896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:37175: BlockingRecv error: recv got EOF from 127.21.27.130:37175 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:35 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:35.891752 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:35.881773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:56497)
0809 22:44:35.882317 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:35.882333 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:35.882486 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:35.882859 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:35.882864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:35.883341 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:35.883525 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.883543 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.885079 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.885091 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.886162 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:35.886174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.886880 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.886884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:35.887020 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.887787 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:35.887809 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:35.891443 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":326,"threads_started":1}
W20260809 22:44:35.891899 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:35.881866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:56497 (local address 127.21.27.190:35739)
0809 22:44:35.882106 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:35.882115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:35.882418 (+ 303us) server_negotiation.cc:407] Connection header received
0809 22:44:35.882633 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:35.882637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:35.882699 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:35.882807 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:35.883742 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.884914 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.886340 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:35.886751 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:35.886847 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:35.891678 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:56497: BlockingRecv error: recv got EOF from 127.21.27.131:56497 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:36.035683 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:36.026394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:36753)
0809 22:44:36.026921 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:36.026938 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:36.027055 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:36.027481 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:36.027487 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:36.027959 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:36.028114 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.028123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.029199 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.029206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.030146 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.030157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.030856 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.030862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.031045 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.031787 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:36.031804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:36.035345 (+ 3541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":294,"threads_started":1}
W20260809 22:44:36.035840 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:36.026507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36753 (local address 127.21.27.190:35739)
0809 22:44:36.026720 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:36.026724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:36.027030 (+ 306us) server_negotiation.cc:407] Connection header received
0809 22:44:36.027231 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:36.027235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:36.027314 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:36.027409 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:36.028266 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.029042 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.030299 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.030697 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.030835 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.035594 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:36753: BlockingRecv error: recv got EOF from 127.21.27.129:36753 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:36.401211 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:36.386990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:60643)
0809 22:44:36.387593 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:36.387610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:36.387811 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:36.388108 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:36.388111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:36.388542 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:36.388699 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.388707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.389735 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.389740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.390756 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.390768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.391807 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.391817 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.392026 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.397312 (+ 5286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:36.397332 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:36.400964 (+ 3632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":363,"threads_started":1}
W20260809 22:44:36.401525 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:36.387316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:60643 (local address 127.21.27.190:35739)
0809 22:44:36.387529 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:36.387538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:36.387669 (+ 131us) server_negotiation.cc:407] Connection header received
0809 22:44:36.387921 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:36.387924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:36.387985 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:36.388071 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:36.388821 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.389587 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.390895 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.391614 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.391755 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.401204 (+ 9449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:60643: BlockingRecv error: recv got EOF from 127.21.27.130:60643 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:36 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:36.902107 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:36.892661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:33171)
0809 22:44:36.893185 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:36.893202 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:36.893358 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:36.893736 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:36.893742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:36.894204 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:36.894354 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.894364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.895462 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.895468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.896537 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:36.896550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.897313 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.897319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:36.897499 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.898147 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:36.898165 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:36.901700 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":277,"threads_started":1}
W20260809 22:44:36.902165 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:36.892730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:33171 (local address 127.21.27.190:35739)
0809 22:44:36.892929 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:36.892936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:36.893290 (+ 354us) server_negotiation.cc:407] Connection header received
0809 22:44:36.893493 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:36.893500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:36.893568 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:36.893697 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:36.894508 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.895329 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.896687 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:36.897120 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:36.897329 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:36.901925 (+ 4596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:33171: BlockingRecv error: recv got EOF from 127.21.27.131:33171 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:37.046325 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:37.036745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:58995)
0809 22:44:37.037402 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:37.037431 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:37.037556 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:37.037887 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:37.037892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:37.038327 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:37.038479 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.038487 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.039944 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.039950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.041052 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.041064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.041775 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.041780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.041906 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.042601 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:37.042620 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:37.046092 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":400,"threads_started":1}
W20260809 22:44:37.046550 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:37.036879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:58995 (local address 127.21.27.190:35739)
0809 22:44:37.037111 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:37.037116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:37.037541 (+ 425us) server_negotiation.cc:407] Connection header received
0809 22:44:37.037652 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:37.037656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:37.037725 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:37.037826 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:37.038650 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.039784 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.041199 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.041640 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.041737 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.046315 (+ 4578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:58995: BlockingRecv error: recv got EOF from 127.21.27.129:58995 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:37.411017 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:37.402159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:37463)
0809 22:44:37.402716 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:37.402739 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:37.402866 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:37.403177 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:37.403185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:37.403636 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:37.403810 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.403819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.404777 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.404781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.405689 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.405701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.406423 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.406427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.406550 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.407176 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:37.407211 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:37.410684 (+ 3473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":265,"threads_started":1}
W20260809 22:44:37.411469 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:37.402244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:37463 (local address 127.21.27.190:35739)
0809 22:44:37.402494 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:37.402502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:37.402831 (+ 329us) server_negotiation.cc:407] Connection header received
0809 22:44:37.402957 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:37.402960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:37.403020 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:37.403113 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:37.403935 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.404651 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.405831 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.406284 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.406460 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.410945 (+ 4485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:37463: BlockingRecv error: recv got EOF from 127.21.27.130:37463 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:37 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:37.912406 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:37.903136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:53465)
0809 22:44:37.903669 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:37.903686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:37.903841 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:37.904198 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:37.904202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:37.904663 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:37.904820 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.904829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.905846 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.905855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.907076 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:37.907093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.908009 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.908015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:37.908203 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.908759 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:37.908779 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:37.912172 (+ 3393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":276,"threads_started":1}
W20260809 22:44:37.912590 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:37.903254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:53465 (local address 127.21.27.190:35739)
0809 22:44:37.903484 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:37.903492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:37.903769 (+ 277us) server_negotiation.cc:407] Connection header received
0809 22:44:37.903962 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:37.903968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:37.904040 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:37.904163 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:37.904962 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.905712 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.907285 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:37.907852 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:37.907971 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:37.912366 (+ 4395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:53465: BlockingRecv error: recv got EOF from 127.21.27.131:53465 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:38.056667 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:38.047469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:36415)
0809 22:44:38.048062 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:38.048078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:38.048237 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:38.048574 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:38.048579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:38.049051 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:38.049311 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.049328 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.050366 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.050372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.051402 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.051413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.052128 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.052134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.052315 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.052857 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:38.052874 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:38.056290 (+ 3416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":257,"threads_started":1}
W20260809 22:44:38.056707 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:38.047535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36415 (local address 127.21.27.190:35739)
0809 22:44:38.047720 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:38.047745 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:38.048170 (+ 425us) server_negotiation.cc:407] Connection header received
0809 22:44:38.048362 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:38.048367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:38.048443 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:38.048545 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:38.049462 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.050223 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.051538 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.051971 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.052113 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.056486 (+ 4373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:36415: BlockingRecv error: recv got EOF from 127.21.27.129:36415 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:38.420796 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:38.412114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:45749)
0809 22:44:38.412847 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:38.412864 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:38.412991 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:38.413281 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:38.413284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:38.413677 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:38.413851 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.413859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.414852 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.414855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.415835 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.415847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.416536 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.416540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.416650 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.417209 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:38.417226 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:38.420539 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":327,"threads_started":1}
W20260809 22:44:38.421141 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:38.412195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:45749 (local address 127.21.27.190:35739)
0809 22:44:38.412412 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:38.412417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:38.412956 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:44:38.413083 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:38.413086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:38.413150 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:38.413232 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:38.413975 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.414725 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.415955 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.416407 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.416499 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.420796 (+ 4297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:45749: BlockingRecv error: recv got EOF from 127.21.27.130:45749 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:38 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:38.922957 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:38.913354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:34277)
0809 22:44:38.913980 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:38.913996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:38.914168 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:38.914532 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:38.914537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:38.914984 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:38.915283 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.915295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.916324 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.916330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.917281 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:38.917293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.918015 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.918029 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:38.918199 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.918802 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:38.918820 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:38.922644 (+ 3824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":309,"threads_started":1}
W20260809 22:44:38.923326 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:38.913443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:34277 (local address 127.21.27.190:35739)
0809 22:44:38.913712 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:38.913721 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:38.914095 (+ 374us) server_negotiation.cc:407] Connection header received
0809 22:44:38.914294 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:38.914306 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:38.914369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:38.914477 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:38.915425 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.916184 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.917425 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:38.917858 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:38.917999 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:38.922889 (+ 4890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:34277: BlockingRecv error: recv got EOF from 127.21.27.131:34277 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:39.067073 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:39.057807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:53179)
0809 22:44:39.058547 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:39.058564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:39.058676 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:39.059030 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:39.059033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:39.059479 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:39.059664 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.059675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.060704 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.060708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.061677 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.061690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.062502 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.062506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.062621 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.063315 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:39.063333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:39.066817 (+ 3484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":485,"threads_started":1}
W20260809 22:44:39.067310 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:39.057961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:53179 (local address 127.21.27.190:35739)
0809 22:44:39.058221 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:39.058230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:39.058651 (+ 421us) server_negotiation.cc:407] Connection header received
0809 22:44:39.058825 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:39.058829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:39.058896 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:39.058987 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:39.059805 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.060553 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.061828 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.062341 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.062438 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.067016 (+ 4578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:53179: BlockingRecv error: recv got EOF from 127.21.27.129:53179 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:39.432137 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:39.421971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:53165)
0809 22:44:39.422712 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:39.422737 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:39.422901 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:39.423363 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:39.423371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:39.423822 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:39.423992 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.424000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.425366 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.425372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.426640 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.426654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.427487 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.427493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.427671 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.428351 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:39.428369 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:39.431873 (+ 3504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":320,"threads_started":1}
W20260809 22:44:39.432360 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:39.422144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:53165 (local address 127.21.27.190:35739)
0809 22:44:39.422392 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:39.422401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:39.422858 (+ 457us) server_negotiation.cc:407] Connection header received
0809 22:44:39.423101 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:39.423106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:39.423200 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:39.423290 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:39.424132 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.425197 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.426783 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.427299 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.427455 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.432121 (+ 4666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:53165: BlockingRecv error: recv got EOF from 127.21.27.130:53165 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:39 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:39.933969 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:39.923964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:59431)
0809 22:44:39.924516 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:39.924533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:39.924698 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:39.925110 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:39.925115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:39.925567 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:39.925730 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.925740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.926857 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.926863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.927879 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:39.927892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.928736 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.928743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:39.928905 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.929600 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:39.929617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:39.933616 (+ 3999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":290,"threads_started":1}
W20260809 22:44:39.934089 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:39.924022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:59431 (local address 127.21.27.190:35739)
0809 22:44:39.924265 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:39.924274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:39.924630 (+ 356us) server_negotiation.cc:407] Connection header received
0809 22:44:39.924834 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:39.924843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:39.924907 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:39.925015 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:39.925891 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.926710 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.928027 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:39.928501 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:39.928673 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:39.933828 (+ 5155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:59431: BlockingRecv error: recv got EOF from 127.21.27.131:59431 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:40.078033 31685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:40.068181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:56525)
0809 22:44:40.068756 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:40.068774 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:40.068946 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:40.069328 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:40.069333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:40.069769 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:40.069915 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.069925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.071026 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.071031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.072060 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.072073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.072847 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.072851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.073001 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.073679 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:40.073698 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:40.077634 (+ 3936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":317,"threads_started":1}
W20260809 22:44:40.078146 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:40.068309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:56525 (local address 127.21.27.190:35739)
0809 22:44:40.068590 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:40.068599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:40.068873 (+ 274us) server_negotiation.cc:407] Connection header received
0809 22:44:40.069094 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:40.069104 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:40.069164 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:40.069271 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:40.070090 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.070864 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.072209 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.072673 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.072811 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.077892 (+ 5081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:56525: BlockingRecv error: recv got EOF from 127.21.27.129:56525 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:40.443719 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:40.433304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:39959)
0809 22:44:40.433841 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:40.433857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:40.434032 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:40.434384 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:40.434388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:40.434842 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:40.434984 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.434992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.436085 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.436092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.437038 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.437049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.437919 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.437926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.438148 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.439056 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:40.439079 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:40.443391 (+ 4312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":294,"threads_started":1}
W20260809 22:44:40.443924 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:40.433343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:39959 (local address 127.21.27.190:35739)
0809 22:44:40.433570 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:40.433579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:40.433957 (+ 378us) server_negotiation.cc:407] Connection header received
0809 22:44:40.434169 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:40.434174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:40.434248 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:40.434353 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:40.435167 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.435944 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.437194 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.437730 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.437887 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.443658 (+ 5771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:39959: BlockingRecv error: recv got EOF from 127.21.27.130:39959 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:40 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:40.944530 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:40.935006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:33201 (local address 127.21.27.190:35739)
0809 22:44:40.935243 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:40.935247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:40.935637 (+ 390us) server_negotiation.cc:407] Connection header received
0809 22:44:40.935781 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:40.935784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:40.935848 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:40.935935 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:40.936706 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.937443 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.938712 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.939228 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.939334 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.944276 (+ 4942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:33201: BlockingRecv error: recv got EOF from 127.21.27.131:33201 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:44:40.944914 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:40.934970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:33201)
0809 22:44:40.935545 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:40.935562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:40.935676 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:40.935991 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:40.935994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:40.936418 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:40.936575 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.936584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.937582 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.937586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.938569 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:40.938581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:40.939370 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:40.939375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:40.939496 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:40.940085 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:40.940103 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:40.944597 (+ 4494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":320,"threads_started":1}
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:41.088418 31688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:41.079044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:36061)
0809 22:44:41.079555 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:41.079573 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:41.079726 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:41.080117 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:41.080122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:41.080522 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:41.080678 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.080686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.081705 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.081711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.082706 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.082718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.083570 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.083576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.083755 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.084317 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:41.084334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:41.088020 (+ 3686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":275,"threads_started":1}
W20260809 22:44:41.088516 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:41.079188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36061 (local address 127.21.27.190:35739)
0809 22:44:41.079414 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:41.079420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:41.079656 (+ 236us) server_negotiation.cc:407] Connection header received
0809 22:44:41.079867 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:41.079872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:41.079971 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:41.080069 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:41.080824 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.081554 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.082871 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.083411 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.083555 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.088231 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:36061: BlockingRecv error: recv got EOF from 127.21.27.129:36061 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:41.459657 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:41.444999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:53077)
0809 22:44:41.445708 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:41.445725 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:41.445834 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:41.447228 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:41.447233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:41.447669 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:41.447839 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.447849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.448898 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.448904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.450121 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.450136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.450899 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.450905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.451079 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.455461 (+ 4382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:41.455483 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:41.459376 (+ 3893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":431,"threads_started":1}
W20260809 22:44:41.460079 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:41.446724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:53077 (local address 127.21.27.190:35739)
0809 22:44:41.446904 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:41.446908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:41.446925 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:44:41.446977 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:41.446980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:41.447040 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:41.447179 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:41.448003 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.448749 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.450280 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.450736 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.450880 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.459681 (+ 8801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:53077: BlockingRecv error: recv got EOF from 127.21.27.130:53077 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:41 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:41.955900 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:41.946010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:37003)
0809 22:44:41.946613 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:41.946632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:41.946755 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:41.947093 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:41.947096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:41.947484 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:41.947658 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.947667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.948923 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.948930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.950169 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:41.950183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.950965 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.950970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:41.951100 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.951861 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:41.951897 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:41.955636 (+ 3739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":339,"threads_started":1}
W20260809 22:44:41.956199 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:41.946333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:37003 (local address 127.21.27.190:35739)
0809 22:44:41.946610 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:41.946614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:41.946715 (+ 101us) server_negotiation.cc:407] Connection header received
0809 22:44:41.946864 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:41.946868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:41.946933 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:41.947032 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:41.947839 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.948762 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.950339 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:41.950806 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:41.950911 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:41.955885 (+ 4974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:37003: BlockingRecv error: recv got EOF from 127.21.27.131:37003 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:44:42.106514 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:42.089646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:52939 (local address 127.21.27.190:35739)
0809 22:44:42.089833 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:42.089838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:42.090151 (+ 313us) server_negotiation.cc:407] Connection header received
0809 22:44:42.090329 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:42.090333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:42.090399 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:42.090492 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:42.091473 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.092945 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.097800 (+ 4855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.098446 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.098588 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.106329 (+ 7741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:52939: BlockingRecv error: recv got EOF from 127.21.27.129:52939 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:44:42.106545 31692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:42.089509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:52939)
0809 22:44:42.090051 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:42.090068 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:42.090185 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:42.090541 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:42.090544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:42.091030 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:42.091292 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.091302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.094091 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.094098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.096071 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.096142 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.098615 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.098621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.098835 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.099771 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:42.099802 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:42.106058 (+ 6256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:42.470489 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:42.460631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:54357)
0809 22:44:42.461178 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:42.461194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:42.461361 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:42.461705 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:42.461710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:42.462205 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:42.462395 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.462404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.463554 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.463559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.464534 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.464547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.465276 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.465283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.465457 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.466144 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:42.466162 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:42.470230 (+ 4068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":292,"threads_started":1}
W20260809 22:44:42.470523 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:42.460761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:54357 (local address 127.21.27.190:35739)
0809 22:44:42.460993 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:42.461001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:42.461284 (+ 283us) server_negotiation.cc:407] Connection header received
0809 22:44:42.461485 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:42.461490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:42.461568 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:42.461674 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:42.462542 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.463419 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.464685 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.465109 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.465243 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.470323 (+ 5080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:54357: BlockingRecv error: recv got EOF from 127.21.27.130:54357 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:42 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:42.965981 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:42.956883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:34435)
0809 22:44:42.957454 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:42.957471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:42.957598 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:42.957895 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:42.957899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:42.958331 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:42.958527 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.958536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.959630 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.959634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.960757 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:42.960777 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.961596 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.961600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:42.961816 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.962460 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:42.962477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:42.965759 (+ 3282us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":332,"threads_started":1}
W20260809 22:44:42.966148 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:42.956971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:34435 (local address 127.21.27.190:35739)
0809 22:44:42.957149 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:42.957154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:42.957555 (+ 401us) server_negotiation.cc:407] Connection header received
0809 22:44:42.957699 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:42.957703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:42.957764 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:42.957849 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:42.958656 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.959476 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.960926 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:42.961429 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:42.961579 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:42.965925 (+ 4346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:34435: BlockingRecv error: recv got EOF from 127.21.27.131:34435 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:43.116705 31695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:43.107473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:53583)
0809 22:44:43.107982 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:43.107997 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:43.108155 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:43.108504 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:43.108509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:43.108955 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:43.109114 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.109124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.110203 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.110211 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.111234 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.111246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.111976 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.111981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.112182 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.112835 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:43.112852 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:43.116366 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":274,"threads_started":1}
W20260809 22:44:43.116807 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:43.107529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:53583 (local address 127.21.27.190:35739)
0809 22:44:43.107769 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:43.107778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:43.108084 (+ 306us) server_negotiation.cc:407] Connection header received
0809 22:44:43.108298 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:43.108302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:43.108366 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:43.108473 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:43.109268 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.110059 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.111380 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.111827 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.111964 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.116574 (+ 4610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:53583: BlockingRecv error: recv got EOF from 127.21.27.129:53583 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:43.481014 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:43.471539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:38521)
0809 22:44:43.472069 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:43.472085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:43.472253 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:43.472596 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:43.472602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:43.473026 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:43.473203 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.473213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.474314 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.474321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.475366 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.475379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.476083 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.476089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.476290 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.476938 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:43.476956 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:43.480678 (+ 3722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":286,"threads_started":1}
W20260809 22:44:43.481221 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:43.471616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:38521 (local address 127.21.27.190:35739)
0809 22:44:43.471863 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:43.471871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:43.472174 (+ 303us) server_negotiation.cc:407] Connection header received
0809 22:44:43.472379 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:43.472391 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:43.472458 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:43.472562 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:43.473359 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.474178 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.475504 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.475933 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.476069 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.480913 (+ 4844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:38521: BlockingRecv error: recv got EOF from 127.21.27.130:38521 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:43 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:43.977195 31697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:43.967075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:50509)
0809 22:44:43.967808 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:43.967825 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:43.967990 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:43.968378 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:43.968384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:43.968814 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:43.969064 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.969082 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.970661 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.970667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.971822 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:43.971835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.972556 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.972561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:43.972689 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.973352 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:43.973371 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:43.976954 (+ 3583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":467,"threads_started":1}
W20260809 22:44:43.977448 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:43.967245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:50509 (local address 127.21.27.190:35739)
0809 22:44:43.967590 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:43.967599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:43.967914 (+ 315us) server_negotiation.cc:407] Connection header received
0809 22:44:43.968137 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:43.968147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:43.968210 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:43.968322 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:43.969258 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.970477 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.971953 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:43.972383 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:43.972482 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:43.977172 (+ 4690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:50509: BlockingRecv error: recv got EOF from 127.21.27.131:50509 (error 108)
Metrics: {"server-negotiator.queue_time_us":175}
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:44.127416 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:44.117845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:49461)
0809 22:44:44.118438 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:44.118454 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:44.118619 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:44.118963 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:44.118968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:44.119468 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:44.119619 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.119628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.120640 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.120646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.121668 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.121680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.122461 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.122467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.122651 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.123360 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:44.123378 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:44.127046 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":365,"threads_started":1}
W20260809 22:44:44.127611 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:44.117925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:49461 (local address 127.21.27.190:35739)
0809 22:44:44.118202 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:44.118210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:44.118558 (+ 348us) server_negotiation.cc:407] Connection header received
0809 22:44:44.118748 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:44.118759 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:44.118821 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:44.118929 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:44.119765 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.120480 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.121812 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.122312 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.122448 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.127311 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:49461: BlockingRecv error: recv got EOF from 127.21.27.129:49461 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:44.491447 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:44.482329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:32943)
0809 22:44:44.482837 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:44.482856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:44.483056 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:44.483450 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:44.483453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:44.483961 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:44.484140 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.484155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.485184 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.485187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.486100 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.486111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.486833 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.486837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.486961 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.487630 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:44.487665 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:44.491191 (+ 3526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":424,"threads_started":1}
W20260809 22:44:44.491761 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:44.482520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:32943 (local address 127.21.27.190:35739)
0809 22:44:44.482749 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:44.482757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:44.482936 (+ 179us) server_negotiation.cc:407] Connection header received
0809 22:44:44.483215 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:44.483221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:44.483305 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:44.483410 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:44.484304 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.485063 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.486237 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.486708 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.486802 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.491438 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:32943: BlockingRecv error: recv got EOF from 127.21.27.130:32943 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:44 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.131@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:44.987409 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:44.978240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.131:45899)
0809 22:44:44.978812 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:44.978833 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:44.979012 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:44.979435 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:44.979440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:44.980060 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:44.980232 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.980242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.981232 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.981236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.982141 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:44.982153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.982844 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.982848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:44.982977 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.983618 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:44.983635 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:44.987131 (+ 3496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":321,"threads_started":1}
W20260809 22:44:44.987738 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:44.978357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:45899 (local address 127.21.27.190:35739)
0809 22:44:44.978556 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:44.978565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:44.978971 (+ 406us) server_negotiation.cc:407] Connection header received
0809 22:44:44.979183 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:44.979188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:44.979268 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:44.979378 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:44.980362 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.981101 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.982284 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:44.982710 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:44.982804 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:44.987431 (+ 4627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.131:45899: BlockingRecv error: recv got EOF from 127.21.27.131:45899 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 22:44:45 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:45 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.129@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:45.137481 31702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:45.128731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.129:60623)
0809 22:44:45.129239 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:45.129256 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:45.129367 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:45.129642 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:45.129645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:45.130028 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:45.130209 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:45.130217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.131225 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.131229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:45.132162 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:45.132173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.132893 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.132897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:45.133031 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.133660 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:45.133677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:45.137238 (+ 3561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
W20260809 22:44:45.137769 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:45.128774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:60623 (local address 127.21.27.190:35739)
0809 22:44:45.129094 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:45.129101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:45.129336 (+ 235us) server_negotiation.cc:407] Connection header received
0809 22:44:45.129449 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:45.129453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:45.129516 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:45.129598 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:45.130329 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.131049 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.132307 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.132767 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.132869 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.137462 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.129:60623: BlockingRecv error: recv got EOF from 127.21.27.129:60623 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 09 22:44:45 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:44:45 dist-test-slave-chsk krb5kdc[30536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.27.130@KRBTEST.COM for oryx/127.21.27.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:44:45.502579 31703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:45.492737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35739 (local address 127.21.27.130:57535)
0809 22:44:45.493406 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:45.493423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:45.493594 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:45.493948 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:45.493952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:45.494389 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:45.494568 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:45.494577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.495672 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.495680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:45.496767 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:45.496779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.497654 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.497662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:45.497978 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.498663 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:45.498681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:45.502201 (+ 3520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.27.190:35739: Server oryx/127.21.27.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":350,"threads_started":1}
W20260809 22:44:45.502655 31653 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:45.492904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:57535 (local address 127.21.27.190:35739)
0809 22:44:45.493167 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:45.493174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:45.493533 (+ 359us) server_negotiation.cc:407] Connection header received
0809 22:44:45.493727 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:45.493734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:45.493804 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:45.493912 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:45.494716 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.495520 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.496906 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.497483 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.497639 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.502401 (+ 4762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.27.130:57535: BlockingRecv error: recv got EOF from 127.21.27.130:57535 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:44:45.571302 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31187
I20260809 22:44:45.584079 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:45.572354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.21.27.129:41529)
0809 22:44:45.573072 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:45.573079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:45.573114 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:44:45.573175 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:45.573182 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:45.573417 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:45.573575 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:45.575106 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.576212 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.577475 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.578175 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.578276 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.580843 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:45.580851 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:45.580864 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:45.580906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:45.583395 (+ 2489us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:44:45.583757 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47322: Request ticket server kudu/127.21.27.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":362,"threads_started":1}
W20260809 22:44:45.584087 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31187: Not authorized: Client connection negotiation failed: client connection to 127.21.27.129:41529: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:44:45.584168 31705 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47322: Request ticket server kudu/127.21.27.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:44:45.584332 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31187
I20260809 22:44:45.604360 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31358
I20260809 22:44:45.617056 31706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:45.605600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35382 (local address 127.21.27.130:36849)
0809 22:44:45.606217 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:45.606225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:45.606253 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:44:45.606315 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:45.606322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:45.606553 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:45.606762 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:45.608205 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.609275 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.610550 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.611123 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.611336 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.613768 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:45.613776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:45.613788 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:45.613825 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:45.616314 (+ 2489us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:44:45.616694 (+ 380us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35382: Request ticket server kudu/127.21.27.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":267,"threads_started":1}
W20260809 22:44:45.617101 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31358: Not authorized: Client connection negotiation failed: client connection to 127.21.27.130:36849: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:44:45.617193 31706 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35382: Request ticket server kudu/127.21.27.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:44:45.617266 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31358
I20260809 22:44:45.637060 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31508
I20260809 22:44:45.650393 31707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:45.638297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.21.27.131:45371)
0809 22:44:45.639039 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:45.639049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:45.639083 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:44:45.639273 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:45.639288 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:45.639603 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:45.639844 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:45.641146 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.642198 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.643636 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:45.644280 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:45.644415 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:45.647044 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:45.647053 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:45.647067 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:45.647111 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:45.649819 (+ 2708us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 22:44:45.650124 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37438: Request ticket server kudu/127.21.27.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":367,"threads_started":1}
W20260809 22:44:45.650501 31707 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37438: Request ticket server kudu/127.21.27.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 22:44:45.650557 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31508: Not authorized: Client connection negotiation failed: client connection to 127.21.27.131:45371: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.27.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 22:44:45.650753 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31508
I20260809 22:44:45.669883 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31112
I20260809 22:44:45.671394 31180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:45.796417 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31112
2026-08-09T22:44:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21538 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T22:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:45Z Disabled control of system clock
I20260809 22:44:45.882277 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35617
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:35617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:44:46.213593 31713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:46.213892 31713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:46.214071 31713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:46.221264 31713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:46.221370 31713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:46.221464 31713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:46.221570 31713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:46.221633 31713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:46.231375 31713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:35617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:46.232784 31713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:46.234508 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:46.246707 31718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:46.246634 31721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:46.246678 31719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:46.247004 31713 server_base.cc:1034] running on GCE node
I20260809 22:44:46.248143 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:46.249604 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:46.250833 31713 hybrid_clock.cc:648] HybridClock initialized: now 1786315486250788 us; error 38 us; skew 500 ppm
I20260809 22:44:46.251286 31713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:46.253635 31713 webserver.cc:460] Webserver started at http://127.21.27.190:45797/ using document root <none> and password file <none>
I20260809 22:44:46.254333 31713 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:46.254463 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:46.254798 31713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:46.257804 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "54c07bf0202c43208a4ec244329b59e8"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "53badfd4ef8096ba1efcca9402c347e0"
server_key_iv: "1bb3a5ea40f051c975628b06041b3e60"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.258591 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "54c07bf0202c43208a4ec244329b59e8"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "53badfd4ef8096ba1efcca9402c347e0"
server_key_iv: "1bb3a5ea40f051c975628b06041b3e60"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.263600 31713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:44:46.266834 31727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:46.268136 31713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:46.268325 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "54c07bf0202c43208a4ec244329b59e8"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "53badfd4ef8096ba1efcca9402c347e0"
server_key_iv: "1bb3a5ea40f051c975628b06041b3e60"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.268545 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:46.302239 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:46.303278 31713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:46.303582 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:46.322706 31713 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:35617
I20260809 22:44:46.322741 31785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:35617 every 8 connection(s)
I20260809 22:44:46.324060 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:46.327677 31786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:46.334081 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31713
I20260809 22:44:46.334311 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:46.335207 21614 external_mini_cluster.cc:1428] Setting key 7990f5fec5aabc9034d6e0be28e96dca
I20260809 22:44:46.338820 31786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8: Bootstrap starting.
I20260809 22:44:46.342152 31786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:46.343223 31786 log.cc:824] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:46.346199 31786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8: No bootstrap required, opened a new log
I20260809 22:44:46.349103 31790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:46.337315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.21.27.190:35617)
0809 22:44:46.338319 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:46.338329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:46.338370 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:44:46.340120 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:46.340143 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:46.340611 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:46.340909 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:46.341629 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.342996 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.344633 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.345337 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.345465 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:46.346786 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:46.346794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:46.346806 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:46.347400 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:46.347470 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:46.347490 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:46.347650 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:46.347882 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 22:44:46.348166 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":386,"threads_started":1}
I20260809 22:44:46.354451 31786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } }
I20260809 22:44:46.354802 31786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:46.354909 31786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c07bf0202c43208a4ec244329b59e8, State: Initialized, Role: FOLLOWER
I20260809 22:44:46.355521 31786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } }
I20260809 22:44:46.355768 31786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:46.355911 31786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:46.356086 31786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:46.358570 31786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } }
I20260809 22:44:46.359046 31786 leader_election.cc:302] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c07bf0202c43208a4ec244329b59e8; no voters:
I20260809 22:44:46.359670 31786 leader_election.cc:288] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:46.359793 31792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:46.360553 31792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [term 1 LEADER]: Becoming Leader. State: Replica: 54c07bf0202c43208a4ec244329b59e8, State: Running, Role: LEADER
I20260809 22:44:46.361086 31792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } }
I20260809 22:44:46.361732 31786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:46.363973 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c07bf0202c43208a4ec244329b59e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } } }
I20260809 22:44:46.363981 31794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c07bf0202c43208a4ec244329b59e8. Latest consensus state: current_term: 1 leader_uuid: "54c07bf0202c43208a4ec244329b59e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c07bf0202c43208a4ec244329b59e8" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35617 } } }
I20260809 22:44:46.364770 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:46.364791 31794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:46.367306 31797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:46.372370 31797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:46.381263 31797 catalog_manager.cc:1347] Generated new cluster ID: b0df7bfbec0e44b6bc71a5b767f98c6b
I20260809 22:44:46.381379 31797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:46.399833 31797 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:46.401098 31797 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:46.407228 31797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c07bf0202c43208a4ec244329b59e8: Generated new TSK 0
I20260809 22:44:46.408247 31797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:44:46.421975 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--builtin_ntp_servers=127.21.27.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 22:44:46.756330 31811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:46.756644 31811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:46.756827 31811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:46.764216 31811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:46.764338 31811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:46.764441 31811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:46.773904 31811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:46.775672 31811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:46.777515 31811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:46.792090 31820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:46.792114 31818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:46.792344 31817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:46.793213 31811 server_base.cc:1034] running on GCE node
I20260809 22:44:46.793753 31811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:46.794633 31811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:46.795851 31811 hybrid_clock.cc:648] HybridClock initialized: now 1786315486795798 us; error 61 us; skew 500 ppm
I20260809 22:44:46.796146 31811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:46.801785 31811 webserver.cc:460] Webserver started at http://127.21.27.129:45929/ using document root <none> and password file <none>
I20260809 22:44:46.802523 31811 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:46.802663 31811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:46.803007 31811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:46.805946 31811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "feb01d55d4544850af23f6cb0d6634c6"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "9b065e7bc253d42bdf9d747761d73be4"
server_key_iv: "49f1c909d67ec68dd20793500cb7e580"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.806730 31811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "feb01d55d4544850af23f6cb0d6634c6"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "9b065e7bc253d42bdf9d747761d73be4"
server_key_iv: "49f1c909d67ec68dd20793500cb7e580"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.812069 31811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:44:46.815568 31828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:46.816804 31811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:44:46.816983 31811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "feb01d55d4544850af23f6cb0d6634c6"
format_stamp: "Formatted at 2026-08-09 22:44:46 on dist-test-slave-chsk"
server_key: "9b065e7bc253d42bdf9d747761d73be4"
server_key_iv: "49f1c909d67ec68dd20793500cb7e580"
server_key_version: "encryptionkey@0"
I20260809 22:44:46.817217 31811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:46.833259 31811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:46.834112 31811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:46.834424 31811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:46.835449 31811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:46.836879 31811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:46.837002 31811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:46.837134 31811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:46.837217 31811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:46.871572 31811 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:43561
I20260809 22:44:46.871626 31956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:43561 every 8 connection(s)
I20260809 22:44:46.872970 31811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:46.880070 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31811
I20260809 22:44:46.880283 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:46.880950 21614 external_mini_cluster.cc:1428] Setting key b12c7451e879fe01f5b75e5d4bfd11ce
I20260809 22:44:46.883579 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--builtin_ntp_servers=127.21.27.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:44:46.885181 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:46.875503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:45403 (local address 127.21.27.190:35617)
0809 22:44:46.875936 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:46.875940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:46.877129 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 22:44:46.878843 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:46.878849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:46.878934 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:46.879051 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:46.880091 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.880930 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.882415 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.882838 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.882963 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:46.884351 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:46.884357 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:46.884362 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:46.884389 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:46.884429 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:46.884441 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:46.884552 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:46.884752 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 22:44:46.884842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":223,"threads_started":1}
I20260809 22:44:46.886111 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:46.876146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35617 (local address 127.21.27.129:45403)
0809 22:44:46.876976 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:46.877009 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:46.878594 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:46.879290 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:46.879298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:46.879374 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:46.879918 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:46.879933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.881082 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.881087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:46.882272 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:46.882283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:46.883071 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:46.883079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:46.883345 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:46.884048 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:46.884051 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:46.884104 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:46.884172 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:46.884544 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:46.884555 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:46.884565 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:44:46.884741 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 22:44:46.885103 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":262,"threads_started":1}
I20260809 22:44:46.887456 31957 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35617
I20260809 22:44:46.887742 31957 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:46.888429 31957 heartbeater.cc:511] Master 127.21.27.190:35617 requested a full tablet report, sending...
I20260809 22:44:46.890319 31746 ts_manager.cc:194] Registered new tserver with Master: feb01d55d4544850af23f6cb0d6634c6 (127.21.27.129:43561)
I20260809 22:44:46.891814 31746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.129:45403
W20260809 22:44:47.211488 31961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:47.211748 31961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:47.211830 31961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:47.218819 31961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:47.218909 31961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:47.219009 31961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:47.227705 31961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:47.228909 31961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:47.230584 31961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:47.243220 31969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:47.243278 31973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:47.243299 31970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:47.243954 31961 server_base.cc:1034] running on GCE node
I20260809 22:44:47.244485 31961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:47.245402 31961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:47.246591 31961 hybrid_clock.cc:648] HybridClock initialized: now 1786315487246534 us; error 56 us; skew 500 ppm
I20260809 22:44:47.246857 31961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:47.249037 31961 webserver.cc:460] Webserver started at http://127.21.27.130:35599/ using document root <none> and password file <none>
I20260809 22:44:47.249658 31961 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:47.249742 31961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:47.250010 31961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:47.252733 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "ecff63ce0c344e5baac8c1385a4983cd"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "8afef4f61c25333b327b86ba1a66f02f"
server_key_iv: "0c2fafb4ecb024f21255e61e9bece8db"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.253502 31961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "ecff63ce0c344e5baac8c1385a4983cd"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "8afef4f61c25333b327b86ba1a66f02f"
server_key_iv: "0c2fafb4ecb024f21255e61e9bece8db"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.258559 31961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:44:47.262388 31979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.263737 31961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:44:47.263898 31961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "ecff63ce0c344e5baac8c1385a4983cd"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "8afef4f61c25333b327b86ba1a66f02f"
server_key_iv: "0c2fafb4ecb024f21255e61e9bece8db"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.264113 31961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:47.281971 31961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:47.282804 31961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:47.283095 31961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:47.284051 31961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:47.285396 31961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:47.285521 31961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.285656 31961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:47.285759 31961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.318267 31961 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:37059
I20260809 22:44:47.318331 32101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:37059 every 8 connection(s)
I20260809 22:44:47.319653 31961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:47.320427 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 31961
I20260809 22:44:47.320629 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:47.321146 21614 external_mini_cluster.cc:1428] Setting key a0d4dedc360f19111851ac90304cda05
I20260809 22:44:47.325454 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--builtin_ntp_servers=127.21.27.148:36577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 22:44:47.335332 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.323180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:58371 (local address 127.21.27.190:35617)
0809 22:44:47.323381 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:47.323390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:47.324369 (+ 979us) server_negotiation.cc:407] Connection header received
0809 22:44:47.326923 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:47.326928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:47.327005 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:47.327105 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:47.328448 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.329589 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.332026 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.332659 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.332777 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.334443 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:47.334449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:47.334454 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:47.334476 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:47.334535 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:47.334558 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:47.334650 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:47.334900 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 22:44:47.334998 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:44:47.336534 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.323014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35617 (local address 127.21.27.130:58371)
0809 22:44:47.324205 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:47.324257 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:47.326641 (+ 2384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:47.327369 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:47.327382 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:47.327494 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:47.328262 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:47.328288 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.329746 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.329754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:47.331871 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:47.331894 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.332891 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.332900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:47.333225 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.334073 (+ 848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:47.334079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:47.334130 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:47.334229 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:47.334660 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:47.334672 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:47.334682 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:44:47.334857 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 22:44:47.335290 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":439,"threads_started":1}
I20260809 22:44:47.337873 32102 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35617
I20260809 22:44:47.338191 32102 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:47.338958 32102 heartbeater.cc:511] Master 127.21.27.190:35617 requested a full tablet report, sending...
I20260809 22:44:47.340405 31746 ts_manager.cc:194] Registered new tserver with Master: ecff63ce0c344e5baac8c1385a4983cd (127.21.27.130:37059)
I20260809 22:44:47.341260 31746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.130:58371
W20260809 22:44:47.663785 32106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:47.664114 32106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:47.664263 32106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:47.671819 32106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:47.671945 32106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:47.672091 32106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:47.681175 32106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:35617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:47.682477 32106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:47.684242 32106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:47.696962 32112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:47.696997 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:47.697069 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:47.698088 32106 server_base.cc:1034] running on GCE node
I20260809 22:44:47.698724 32106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:47.699801 32106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:47.701030 32106 hybrid_clock.cc:648] HybridClock initialized: now 1786315487700985 us; error 30 us; skew 500 ppm
I20260809 22:44:47.701366 32106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:47.703701 32106 webserver.cc:460] Webserver started at http://127.21.27.131:37601/ using document root <none> and password file <none>
I20260809 22:44:47.704402 32106 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:47.704531 32106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:47.704849 32106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:47.707662 32106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "85e171d919964f8cb2e7318e50b75e58"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "a6d717e08d78fd8d7f43b0795764f3e6"
server_key_iv: "19dc95e6ad70cff2e8665aba892217bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.708431 32106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "85e171d919964f8cb2e7318e50b75e58"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "a6d717e08d78fd8d7f43b0795764f3e6"
server_key_iv: "19dc95e6ad70cff2e8665aba892217bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.713814 32106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:44:47.717423 32121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.718724 32106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:44:47.718961 32106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "85e171d919964f8cb2e7318e50b75e58"
format_stamp: "Formatted at 2026-08-09 22:44:47 on dist-test-slave-chsk"
server_key: "a6d717e08d78fd8d7f43b0795764f3e6"
server_key_iv: "19dc95e6ad70cff2e8665aba892217bc"
server_key_version: "encryptionkey@0"
I20260809 22:44:47.719209 32106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:47.745870 32106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:47.746714 32106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:47.747014 32106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:47.747974 32106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:47.749380 32106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:47.749505 32106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.749629 32106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:47.749712 32106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:47.782912 32106 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:36351
I20260809 22:44:47.782994 32244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:36351 every 8 connection(s)
I20260809 22:44:47.784209 32106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:47.785429 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 32106
I20260809 22:44:47.785619 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:47.786198 21614 external_mini_cluster.cc:1428] Setting key 8cfd3dcaa752d7a755699a537d4ed9cc
I20260809 22:44:47.796347 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.786918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:57077 (local address 127.21.27.190:35617)
0809 22:44:47.787108 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:47.787113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:47.788811 (+ 1698us) server_negotiation.cc:407] Connection header received
0809 22:44:47.790687 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:47.790691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:47.790756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:47.790837 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:47.791823 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.792505 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.793946 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.794344 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.794429 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.795707 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:47.795712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:47.795715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:47.795732 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:47.795761 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:47.795772 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:47.795840 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:47.796022 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 22:44:47.796106 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:44:47.797245 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.787549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:35617 (local address 127.21.27.131:57077)
0809 22:44:47.788630 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:47.788688 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:47.790459 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:47.791012 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:47.791019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:47.791130 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:47.791695 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:47.791710 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.792632 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.792636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:47.793831 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:47.793843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.794510 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.794514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:47.794692 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.795419 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:47.795422 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:47.795456 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:47.795525 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:47.795850 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:47.795856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:47.795861 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:44:47.795985 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 22:44:47.796254 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":303,"threads_started":1}
I20260809 22:44:47.798516 32245 heartbeater.cc:347] Connected to a master server at 127.21.27.190:35617
I20260809 22:44:47.798804 32245 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:47.799546 32245 heartbeater.cc:511] Master 127.21.27.190:35617 requested a full tablet report, sending...
I20260809 22:44:47.800921 31746 ts_manager.cc:194] Registered new tserver with Master: 85e171d919964f8cb2e7318e50b75e58 (127.21.27.131:36351)
I20260809 22:44:47.801738 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:44:47.801867 31746 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.131:57077
W20260809 22:44:47.808017 31959 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:47.806464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44864 (local address 127.21.27.190:35617)
0809 22:44:47.806701 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:47.806709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:47.807088 (+ 379us) server_negotiation.cc:407] Connection header received
0809 22:44:47.807267 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:47.807271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:47.807339 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:47.807417 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:47.807764 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44864: BlockingRecv error: recv got EOF from 127.0.0.1:44864 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:44:47.808818 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31811
I20260809 22:44:47.815876 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.809670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47316 (local address 127.21.27.129:43561)
0809 22:44:47.810194 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:47.810199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:47.810222 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:44:47.810279 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:47.810285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:47.810447 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:47.810569 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:47.810923 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.811914 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.813048 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.813563 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.813663 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.814579 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:47.814613 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:47.814624 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:47.815215 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:47.815275 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:47.815293 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:47.815378 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:47.815482 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 22:44:47.815601 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":300,"threads_started":1}
I20260809 22:44:47.817075 31952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:47.897070 31957 heartbeater.cc:503] Master 127.21.27.190:35617 was elected leader, sending a full tablet report...
I20260809 22:44:47.898540 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31811
W20260809 22:44:47.917120 31729 connection.cc:535] server connection from 127.21.27.129:45403 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 22:44:47.917480 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31961
I20260809 22:44:47.925441 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:47.918442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.21.27.130:37059)
0809 22:44:47.918961 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:47.918968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:47.918994 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:44:47.919052 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:47.919059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:47.919340 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:47.919503 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:47.919892 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.920905 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.922148 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:47.922697 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:47.922832 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:47.924038 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:47.924048 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:47.924059 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:47.924633 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:47.924676 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:47.924695 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:47.924817 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:47.924959 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 22:44:47.925103 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":263,"threads_started":1}
I20260809 22:44:47.926791 32097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:48.008440 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31961
I20260809 22:44:48.027555 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 32106
I20260809 22:44:48.036545 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:48.028534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.21.27.131:36351)
0809 22:44:48.029235 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:48.029243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:48.029275 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:44:48.029338 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:48.029344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:48.029584 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:48.029769 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:48.030301 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:48.031537 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:48.032969 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:48.033594 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:48.033752 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:48.034855 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:48.034864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:48.034876 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:48.035658 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:48.035716 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:48.035737 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:48.035858 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:48.036027 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 22:44:48.036160 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":371,"threads_started":1}
I20260809 22:44:48.037777 32240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:48.119222 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 32106
I20260809 22:44:48.136860 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 31713
I20260809 22:44:48.138171 31780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:48.241120 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 31713
2026-08-09T22:44:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2439 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T22:44:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:48Z Disabled control of system clock
I20260809 22:44:48.324039 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:46719
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:46719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:44:48.655004 32267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:48.655448 32267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:48.655627 32267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:48.663188 32267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:48.663287 32267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:48.663332 32267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:48.663396 32267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:48.663441 32267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:48.672335 32267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:46719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:46719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:48.673555 32267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:48.675248 32267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:48.686542 32275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:48.686646 32272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:48.686657 32273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:48.687368 32267 server_base.cc:1034] running on GCE node
I20260809 22:44:48.688033 32267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:48.689242 32267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:48.690466 32267 hybrid_clock.cc:648] HybridClock initialized: now 1786315488690429 us; error 48 us; skew 500 ppm
I20260809 22:44:48.690797 32267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:48.693001 32267 webserver.cc:460] Webserver started at http://127.21.27.190:46217/ using document root <none> and password file <none>
I20260809 22:44:48.693645 32267 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:48.693769 32267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:48.694093 32267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:48.696882 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "ce235bc020884b0ea21db1705441b25c"
format_stamp: "Formatted at 2026-08-09 22:44:48 on dist-test-slave-chsk"
server_key: "035b00c4f88ab2ca3a7895e1bb6d712f"
server_key_iv: "1895a821a31a3c39d4c7c5674a045273"
server_key_version: "encryptionkey@0"
I20260809 22:44:48.697734 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "ce235bc020884b0ea21db1705441b25c"
format_stamp: "Formatted at 2026-08-09 22:44:48 on dist-test-slave-chsk"
server_key: "035b00c4f88ab2ca3a7895e1bb6d712f"
server_key_iv: "1895a821a31a3c39d4c7c5674a045273"
server_key_version: "encryptionkey@0"
I20260809 22:44:48.702512 32267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:44:48.705757 32281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:48.707007 32267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:44:48.707226 32267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "ce235bc020884b0ea21db1705441b25c"
format_stamp: "Formatted at 2026-08-09 22:44:48 on dist-test-slave-chsk"
server_key: "035b00c4f88ab2ca3a7895e1bb6d712f"
server_key_iv: "1895a821a31a3c39d4c7c5674a045273"
server_key_version: "encryptionkey@0"
I20260809 22:44:48.707441 32267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:48.712827 32267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:48.713563 32267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:48.713888 32267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:48.732530 32267 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:46719
I20260809 22:44:48.732627 32336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:46719 every 8 connection(s)
I20260809 22:44:48.734169 32267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:48.737903 32338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:48.738420 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 32267
I20260809 22:44:48.738665 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:48.739526 21614 external_mini_cluster.cc:1428] Setting key 29712aeed2a098e01052bfcb91475b05
I20260809 22:44:48.747975 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:48.741671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.21.27.190:46719)
0809 22:44:48.742659 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:48.742669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:48.742714 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:44:48.744527 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:48.744548 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:48.744951 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:48.745283 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:48.746358 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:48.746364 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:48.746375 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:48.746389 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:48.746450 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:48.746466 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:48.746638 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:48.746753 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 22:44:48.747083 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":389,"threads_started":1}
I20260809 22:44:48.751024 32338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c: Bootstrap starting.
I20260809 22:44:48.753840 32338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:48.754797 32338 log.cc:824] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:48.757561 32338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c: No bootstrap required, opened a new log
I20260809 22:44:48.764911 32338 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } }
I20260809 22:44:48.765278 32338 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:48.765381 32338 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce235bc020884b0ea21db1705441b25c, State: Initialized, Role: FOLLOWER
I20260809 22:44:48.765901 32338 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } }
I20260809 22:44:48.766141 32338 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:48.766280 32338 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:48.766443 32338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:48.768694 32338 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } }
I20260809 22:44:48.769160 32338 leader_election.cc:302] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce235bc020884b0ea21db1705441b25c; no voters:
I20260809 22:44:48.769846 32338 leader_election.cc:288] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:48.769972 32343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:48.770740 32343 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [term 1 LEADER]: Becoming Leader. State: Replica: ce235bc020884b0ea21db1705441b25c, State: Running, Role: LEADER
I20260809 22:44:48.771361 32343 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } }
I20260809 22:44:48.772284 32338 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:48.774345 32345 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce235bc020884b0ea21db1705441b25c. Latest consensus state: current_term: 1 leader_uuid: "ce235bc020884b0ea21db1705441b25c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } } }
I20260809 22:44:48.774770 32345 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:48.777184 32352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:48.777617 32344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce235bc020884b0ea21db1705441b25c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce235bc020884b0ea21db1705441b25c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46719 } } }
I20260809 22:44:48.777918 32344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:48.781600 32352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:48.789758 32352 catalog_manager.cc:1347] Generated new cluster ID: 0b1a6c47d36844269fd3b573a0ab1a7f
I20260809 22:44:48.789857 32352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:48.805629 32352 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:48.806154 32352 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:48.812826 32352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce235bc020884b0ea21db1705441b25c: Generated new TSK 0
I20260809 22:44:48.813656 32352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:44:48.821803 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--builtin_ntp_servers=127.21.27.148:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 22:44:49.154914 32363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:49.155303 32363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:49.155469 32363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:49.163036 32363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:49.163196 32363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:49.163383 32363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:49.172724 32363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:49.174113 32363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:49.176018 32363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:49.188910 32368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:49.188937 32369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:49.189076 32371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:49.189857 32363 server_base.cc:1034] running on GCE node
I20260809 22:44:49.190430 32363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:49.191427 32363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:49.192649 32363 hybrid_clock.cc:648] HybridClock initialized: now 1786315489192607 us; error 47 us; skew 500 ppm
I20260809 22:44:49.192956 32363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:49.195124 32363 webserver.cc:460] Webserver started at http://127.21.27.129:41809/ using document root <none> and password file <none>
I20260809 22:44:49.195837 32363 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:49.195967 32363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:49.196285 32363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:49.201184 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "6394eb71e78e4f9294b06c190de57014"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "b02ff877746aa6a4d9146cdb27a1e455"
server_key_iv: "56563bcdc495a6a9af4bf64e66cda8a5"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.201967 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "6394eb71e78e4f9294b06c190de57014"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "b02ff877746aa6a4d9146cdb27a1e455"
server_key_iv: "56563bcdc495a6a9af4bf64e66cda8a5"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.206812 32363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 22:44:49.210101 32377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.211519 32363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:44:49.211673 32363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "6394eb71e78e4f9294b06c190de57014"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "b02ff877746aa6a4d9146cdb27a1e455"
server_key_iv: "56563bcdc495a6a9af4bf64e66cda8a5"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.211884 32363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:49.217316 32363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:49.218055 32363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:49.218320 32363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:49.219229 32363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:49.220518 32363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:49.220626 32363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.220749 32363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:49.220839 32363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.252516 32363 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:34231
I20260809 22:44:49.252588 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:34231 every 8 connection(s)
I20260809 22:44:49.253806 32363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:49.260322 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 32363
I20260809 22:44:49.260551 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:49.261220 21614 external_mini_cluster.cc:1428] Setting key 9a05d25d5e408c8ef33e46f10d8bce7f
I20260809 22:44:49.262226 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:49.256330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:58719 (local address 127.21.27.190:46719)
0809 22:44:49.256744 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:49.256749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:49.257943 (+ 1194us) server_negotiation.cc:407] Connection header received
0809 22:44:49.259763 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:49.259767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:49.259852 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:49.259998 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:49.261435 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:49.261441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:49.261446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:49.261470 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:49.261522 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:49.261539 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:49.261679 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:49.261872 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 22:44:49.261945 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":195,"threads_started":1}
I20260809 22:44:49.263383 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:49.256909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:46719 (local address 127.21.27.129:58719)
0809 22:44:49.257787 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:49.257822 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:49.259428 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:49.260175 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:49.260184 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:49.260272 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:49.261023 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:49.261026 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:49.261071 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:49.261161 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:49.261665 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:49.261674 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:49.261680 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:44:49.261922 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 22:44:49.262216 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":268,"threads_started":1}
I20260809 22:44:49.264408 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--builtin_ntp_servers=127.21.27.148:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:44:49.264870 32502 heartbeater.cc:347] Connected to a master server at 127.21.27.190:46719
I20260809 22:44:49.265165 32502 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:49.265733 32502 heartbeater.cc:511] Master 127.21.27.190:46719 requested a full tablet report, sending...
I20260809 22:44:49.267755 32299 ts_manager.cc:194] Registered new tserver with Master: 6394eb71e78e4f9294b06c190de57014 (127.21.27.129:34231)
W20260809 22:44:49.590512 32507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:49.590976 32507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:49.591130 32507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:49.604350 32507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:49.604503 32507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:49.604722 32507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:49.620149 32507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:49.621731 32507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:49.623895 32507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:49.639982 32516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:49.639971 32513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:49.640455 32514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:49.642053 32507 server_base.cc:1034] running on GCE node
I20260809 22:44:49.642751 32507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:49.643841 32507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:49.645085 32507 hybrid_clock.cc:648] HybridClock initialized: now 1786315489645029 us; error 51 us; skew 500 ppm
I20260809 22:44:49.645447 32507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:49.648046 32507 webserver.cc:460] Webserver started at http://127.21.27.130:40295/ using document root <none> and password file <none>
I20260809 22:44:49.648833 32507 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:49.648967 32507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:49.649300 32507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:49.652464 32507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "f34fc09d035a4926b421752ac48116c4"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "aec896f32331da0cc327af6d434f681e"
server_key_iv: "39086944b882207820ced1efbe0a2f05"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.653345 32507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "f34fc09d035a4926b421752ac48116c4"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "aec896f32331da0cc327af6d434f681e"
server_key_iv: "39086944b882207820ced1efbe0a2f05"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.658967 32507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:44:49.662703 32524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.664021 32507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:49.664201 32507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "f34fc09d035a4926b421752ac48116c4"
format_stamp: "Formatted at 2026-08-09 22:44:49 on dist-test-slave-chsk"
server_key: "aec896f32331da0cc327af6d434f681e"
server_key_iv: "39086944b882207820ced1efbe0a2f05"
server_key_version: "encryptionkey@0"
I20260809 22:44:49.664433 32507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:49.670176 32507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:49.671018 32507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:49.671308 32507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:49.672268 32507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:49.673821 32507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:49.673940 32507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.674080 32507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:49.674165 32507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:49.708729 32507 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:33075
I20260809 22:44:49.708817 32646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:33075 every 8 connection(s)
I20260809 22:44:49.710225 32507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:49.711596 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 32507
I20260809 22:44:49.711802 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:49.712258 21614 external_mini_cluster.cc:1428] Setting key 84e2bcd9091bf026e90d854769654234
I20260809 22:44:49.715348 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--builtin_ntp_servers=127.21.27.148:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 22:44:49.721459 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:49.714426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:56995 (local address 127.21.27.190:46719)
0809 22:44:49.714622 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:49.714628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:49.716443 (+ 1815us) server_negotiation.cc:407] Connection header received
0809 22:44:49.719118 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:49.719130 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:49.719248 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:49.719336 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:49.720807 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:49.720811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:49.720815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:49.720831 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:49.720862 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:49.720874 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:49.720965 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:49.721180 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 22:44:49.721232 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:44:49.722743 32649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:49.715035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:46719 (local address 127.21.27.130:56995)
0809 22:44:49.716270 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:49.716327 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:49.718847 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:49.719581 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:49.719594 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:49.719723 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:49.720459 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:49.720465 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:49.720518 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:49.720627 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:49.720966 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:49.720978 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:49.720988 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:44:49.721269 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 22:44:49.721514 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":381,"threads_started":1}
I20260809 22:44:49.724136 32647 heartbeater.cc:347] Connected to a master server at 127.21.27.190:46719
I20260809 22:44:49.724448 32647 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:49.725134 32647 heartbeater.cc:511] Master 127.21.27.190:46719 requested a full tablet report, sending...
I20260809 22:44:49.726626 32299 ts_manager.cc:194] Registered new tserver with Master: f34fc09d035a4926b421752ac48116c4 (127.21.27.130:33075)
W20260809 22:44:50.055289 32650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:50.055603 32650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:50.055768 32650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:50.063321 32650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:50.063459 32650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:50.063640 32650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:50.073103 32650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:46719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:50.074465 32650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:50.076273 32650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:50.088994 32660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:50.089123 32656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:50.089066 32658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:50.089984 32650 server_base.cc:1034] running on GCE node
I20260809 22:44:50.090538 32650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:50.091475 32650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:50.092715 32650 hybrid_clock.cc:648] HybridClock initialized: now 1786315490092674 us; error 46 us; skew 500 ppm
I20260809 22:44:50.093048 32650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:44:50.095465 32650 webserver.cc:460] Webserver started at http://127.21.27.131:44313/ using document root <none> and password file <none>
I20260809 22:44:50.096149 32650 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:50.096279 32650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:50.096598 32650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:50.099458 32650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "074fa1f574444c83a6fb5f507be85a85"
format_stamp: "Formatted at 2026-08-09 22:44:50 on dist-test-slave-chsk"
server_key: "cd4f1dc5babdd9d3ff8c55287825d4db"
server_key_iv: "697e9fa65de9b09571418ea31ebbb3ba"
server_key_version: "encryptionkey@0"
I20260809 22:44:50.100276 32650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "074fa1f574444c83a6fb5f507be85a85"
format_stamp: "Formatted at 2026-08-09 22:44:50 on dist-test-slave-chsk"
server_key: "cd4f1dc5babdd9d3ff8c55287825d4db"
server_key_iv: "697e9fa65de9b09571418ea31ebbb3ba"
server_key_version: "encryptionkey@0"
I20260809 22:44:50.105405 32650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:44:50.108726 32666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:50.110028 32650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:44:50.110210 32650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "074fa1f574444c83a6fb5f507be85a85"
format_stamp: "Formatted at 2026-08-09 22:44:50 on dist-test-slave-chsk"
server_key: "cd4f1dc5babdd9d3ff8c55287825d4db"
server_key_iv: "697e9fa65de9b09571418ea31ebbb3ba"
server_key_version: "encryptionkey@0"
I20260809 22:44:50.110435 32650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:50.116048 32650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:50.116940 32650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:50.117211 32650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:50.118105 32650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:50.119549 32650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:50.119653 32650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:50.119783 32650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:50.119863 32650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:50.152522 32650 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:33003
I20260809 22:44:50.152582 321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:33003 every 8 connection(s)
I20260809 22:44:50.153898 32650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:50.157850 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 32650
I20260809 22:44:50.158052 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:50.158507 21614 external_mini_cluster.cc:1428] Setting key e76537ef9097f3f9d5a67f02520ffef1
I20260809 22:44:50.162595 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:50.156527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:59161 (local address 127.21.27.190:46719)
0809 22:44:50.156721 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:50.156726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:50.158182 (+ 1456us) server_negotiation.cc:407] Connection header received
0809 22:44:50.160324 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:50.160330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:50.160408 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:50.160517 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:50.161943 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:50.161948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:50.161952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:50.161972 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:50.162028 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:50.162045 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:50.162188 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:50.162325 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 22:44:50.162375 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:44:50.163591 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:50.157030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:46719 (local address 127.21.27.131:59161)
0809 22:44:50.157984 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:50.158041 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:50.160061 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:50.160743 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:50.160751 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:50.160846 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:50.161602 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:44:50.161608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:50.161661 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:44:50.161772 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:50.162150 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:50.162157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:50.162162 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:44:50.162375 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 22:44:50.162577 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":302,"threads_started":1}
I20260809 22:44:50.164893 322 heartbeater.cc:347] Connected to a master server at 127.21.27.190:46719
I20260809 22:44:50.165195 322 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:50.165777 322 heartbeater.cc:511] Master 127.21.27.190:46719 requested a full tablet report, sending...
I20260809 22:44:50.167196 32299 ts_manager.cc:194] Registered new tserver with Master: 074fa1f574444c83a6fb5f507be85a85 (127.21.27.131:33003)
I20260809 22:44:50.174119 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:44:50.180510 32504 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:44:50.178924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.21.27.190:46719)
0809 22:44:50.179085 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:50.179089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:50.179645 (+ 556us) server_negotiation.cc:407] Connection header received
0809 22:44:50.179793 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:50.179797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:50.179864 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:50.179947 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:50.180273 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46436: BlockingRecv error: recv got EOF from 127.0.0.1:46436 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 22:44:50.180914 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 32363
I20260809 22:44:50.185109 332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:50.182317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.21.27.129:34231)
0809 22:44:50.182892 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:50.182900 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:50.182925 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:44:50.182988 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:50.182995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:50.183288 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:50.183423 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:50.184297 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:50.184303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:50.184314 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:50.184330 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:50.184366 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:50.184380 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:50.184551 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:50.184613 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 22:44:50.184722 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":297,"threads_started":1}
I20260809 22:44:50.186427 32497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:50.268082 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 32363
I20260809 22:44:50.287518 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 32507
I20260809 22:44:50.291375 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:50.288520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.21.27.130:33075)
0809 22:44:50.289031 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:50.289038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:50.289066 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:44:50.289135 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:50.289142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:50.289352 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:50.289532 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:50.290594 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:50.290601 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:50.290613 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:50.290625 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:50.290664 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:50.290677 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:50.290835 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:50.290924 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:44:50.291044 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":227,"threads_started":1}
I20260809 22:44:50.292645 32642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:50.374624 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 32507
I20260809 22:44:50.392244 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 32650
I20260809 22:44:50.396173 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:50.393313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.21.27.131:33003)
0809 22:44:50.393961 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:50.393968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:50.393999 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:44:50.394062 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:50.394069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:50.394309 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:50.394441 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:50.395528 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:50.395534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:50.395545 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:44:50.395562 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:50.395599 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:44:50.395613 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:50.395746 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:50.395799 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 22:44:50.395880 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":322,"threads_started":1}
I20260809 22:44:50.397449 316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:50.481392 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 32650
I20260809 22:44:50.499544 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 32267
I20260809 22:44:50.500828 32332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:50.603816 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 32267
2026-08-09T22:44:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2363 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:50 dist-test-slave-chsk krb5kdc[341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:50 dist-test-slave-chsk krb5kdc[341](info): set up 2 sockets
Aug 09 22:44:50 dist-test-slave-chsk krb5kdc[341](info): commencing operation
krb5kdc: starting...
W20260809 22:44:52.118927 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.470s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:52 dist-test-slave-chsk krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:44:52.604346 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42869
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:52.952342 354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:52.952688 354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:52.952857 354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:52.960489 354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:44:52.960625 354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:52.960734 354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:52.960801 354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:44:52.960875 354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:44:52.971499 354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:52.972967 354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:52.974356 354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:52.987107 362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:52.987098 359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:52.987442 360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:52.988018 354 server_base.cc:1034] running on GCE node
I20260809 22:44:52.988880 354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:52.990476 354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:52.991731 354 hybrid_clock.cc:648] HybridClock initialized: now 1786315492991683 us; error 42 us; skew 500 ppm
I20260809 22:44:52.992121 354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:52 dist-test-slave-chsk krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:53.000334 354 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:44:53.002066 354 webserver.cc:460] Webserver started at http://127.21.27.190:37613/ using document root <none> and password file <none>
I20260809 22:44:53.002830 354 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:53.002980 354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:53.003350 354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:53.006520 354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "ccd914fbfdbe4562a545df2e781b98a1"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "45f56c208c6112468b7507147eb7a54e"
server_key_iv: "2bc683b4fa8ef9ff5b32422379abb4c0"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.007411 354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "ccd914fbfdbe4562a545df2e781b98a1"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "45f56c208c6112468b7507147eb7a54e"
server_key_iv: "2bc683b4fa8ef9ff5b32422379abb4c0"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.012810 354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 22:44:53.016762 370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:53.018275 354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:44:53.018513 354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "ccd914fbfdbe4562a545df2e781b98a1"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "45f56c208c6112468b7507147eb7a54e"
server_key_iv: "2bc683b4fa8ef9ff5b32422379abb4c0"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.018762 354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:53.059494 354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:53.065296 354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:53.065748 354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:53.087824 354 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42869
I20260809 22:44:53.087875 425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42869 every 8 connection(s)
I20260809 22:44:53.089490 354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:44:53.093509 426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:53.095743 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 354
I20260809 22:44:53.095991 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:44:53.096674 21614 external_mini_cluster.cc:1428] Setting key 6fdf460aa64b386ca15f2d3e549d8f64
I20260809 22:44:53.106467 426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: Bootstrap starting.
I20260809 22:44:53.110716 426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:53.112273 426 log.cc:824] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:44:53 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:53.115756 426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: No bootstrap required, opened a new log
I20260809 22:44:53.122776 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:53.099240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.21.27.190:42869)
0809 22:44:53.100188 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:53.100201 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:53.100247 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:44:53.101983 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:53.102018 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:53.102442 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:53.102793 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:53.104873 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.106278 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.108177 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.108932 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.109094 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:53.114545 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:53.114554 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:53.114579 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:53.114624 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:53.118083 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:53.118969 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:53.118980 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:53.118987 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:53.119247 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:53.119676 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:53.119680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:53.119688 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:53.120291 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:53.120676 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:53.121134 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 22:44:53.121643 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":447,"threads_started":1}
I20260809 22:44:53.124835 426 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } }
I20260809 22:44:53.125243 426 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:53.125363 426 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd914fbfdbe4562a545df2e781b98a1, State: Initialized, Role: FOLLOWER
I20260809 22:44:53.126094 426 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } }
I20260809 22:44:53.126379 426 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:44:53.126533 426 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:44:53.126726 426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:53.129403 426 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } }
I20260809 22:44:53.129966 426 leader_election.cc:302] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccd914fbfdbe4562a545df2e781b98a1; no voters:
I20260809 22:44:53.130594 426 leader_election.cc:288] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:44:53.130769 432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:53.131531 432 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [term 1 LEADER]: Becoming Leader. State: Replica: ccd914fbfdbe4562a545df2e781b98a1, State: Running, Role: LEADER
I20260809 22:44:53.132234 432 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } }
I20260809 22:44:53.133190 426 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:44:53.136092 435 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccd914fbfdbe4562a545df2e781b98a1. Latest consensus state: current_term: 1 leader_uuid: "ccd914fbfdbe4562a545df2e781b98a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } } }
I20260809 22:44:53.136091 433 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccd914fbfdbe4562a545df2e781b98a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccd914fbfdbe4562a545df2e781b98a1" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42869 } } }
I20260809 22:44:53.136677 435 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:53.136778 433 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:44:53.138860 438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:44:53.143764 438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:44:53.152833 438 catalog_manager.cc:1347] Generated new cluster ID: fb71db38c01e4f4c865070ba918dde10
I20260809 22:44:53.153013 438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:44:53.162882 438 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:44:53.165011 438 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:44:53.173275 438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: Generated new TSK 0
I20260809 22:44:53.174465 438 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:44:53.248185 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:53.588819 455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:53.589107 455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:53.589260 455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:53.596716 455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:53.596843 455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:53.597010 455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:44:53.607120 455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:53.608525 455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:53.609859 455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:53.623878 463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:53.624106 462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:53.623879 465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:53.625046 455 server_base.cc:1034] running on GCE node
I20260809 22:44:53.625752 455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:53.626839 455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:53.628063 455 hybrid_clock.cc:648] HybridClock initialized: now 1786315493628013 us; error 70 us; skew 500 ppm
I20260809 22:44:53.628407 455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:53 dist-test-slave-chsk krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315493, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:53.631764 455 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:44:53.633378 455 webserver.cc:460] Webserver started at http://127.21.27.129:41175/ using document root <none> and password file <none>
I20260809 22:44:53.634112 455 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:53.634262 455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:53.634575 455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:53.637836 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "f3364ce619c040b7b8cf44b173f9bc82"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "fb4535e5425c08a2e6ced9fe483fdc22"
server_key_iv: "8fed684740aa66b21822a5b69856e6c1"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.638772 455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "f3364ce619c040b7b8cf44b173f9bc82"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "fb4535e5425c08a2e6ced9fe483fdc22"
server_key_iv: "8fed684740aa66b21822a5b69856e6c1"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.644353 455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:44:53.647888 472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:53.649263 455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:44:53.649437 455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "f3364ce619c040b7b8cf44b173f9bc82"
format_stamp: "Formatted at 2026-08-09 22:44:53 on dist-test-slave-chsk"
server_key: "fb4535e5425c08a2e6ced9fe483fdc22"
server_key_iv: "8fed684740aa66b21822a5b69856e6c1"
server_key_version: "encryptionkey@0"
I20260809 22:44:53.649649 455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:53.666555 455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:53.670843 455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:53.671249 455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:53.672384 455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:53.673885 455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:53.673961 455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:53.674109 455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:53.674168 455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:53.708410 455 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:39811
I20260809 22:44:53.708417 591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:39811 every 8 connection(s)
I20260809 22:44:53.709741 455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:44:53.718976 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 455
I20260809 22:44:53.719261 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:44:53.719910 21614 external_mini_cluster.cc:1428] Setting key d16f1fcf68762288cce4f3d46215f608
Aug 09 22:44:53 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315493, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:53.733184 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:53.712806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42869 (local address 127.21.27.129:59189)
0809 22:44:53.713506 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:53.713551 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:53.715257 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:53.715962 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:53.715969 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:53.716490 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:53.717035 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:53.717050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.718110 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.718114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:53.719475 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:53.719487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.720281 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.720288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:53.720602 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:53.721343 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:53.721367 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:53.723960 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:53.728042 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:53.728050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:53.728064 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:53.728823 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:53.729237 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:53.729244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:53.729250 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:53.729560 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:53.730122 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:53.730130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:53.730639 (+ 509us) client_negotiation.cc:769] Sending connection context
0809 22:44:53.731021 (+ 382us) client_negotiation.cc:240] Negotiation successful
0809 22:44:53.731682 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":268,"threads_started":1}
I20260809 22:44:53.734429 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:53.712357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:59189 (local address 127.21.27.190:42869)
0809 22:44:53.712802 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:53.712807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:53.713661 (+ 854us) server_negotiation.cc:407] Connection header received
0809 22:44:53.715521 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:53.715527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:53.715620 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:53.715735 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:53.717207 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.717932 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.719630 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:53.720051 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:53.720192 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:53.724188 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:53.724197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:53.724204 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:53.724250 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:53.727841 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:53.728971 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:53.728975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:53.728978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:53.729111 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:53.729700 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:53.729704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:53.729706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:53.729988 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:53.730152 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:53.733761 (+ 3609us) server_negotiation.cc:299] Negotiation successful
0809 22:44:53.734073 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260809 22:44:53.736845 592 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42869
I20260809 22:44:53.737175 592 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:53.738051 592 heartbeater.cc:511] Master 127.21.27.190:42869 requested a full tablet report, sending...
I20260809 22:44:53.740722 387 ts_manager.cc:194] Registered new tserver with Master: f3364ce619c040b7b8cf44b173f9bc82 (127.21.27.129:39811)
I20260809 22:44:53.742794 387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:59189
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:44:53.786342 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:54.111981 598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:54.112284 598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:54.112452 598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:54.119578 598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:54.119714 598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:54.119913 598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:44:54.129408 598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:54.130657 598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:54.131933 598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:54.144987 608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:54.145051 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:54.145547 604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:54.145737 598 server_base.cc:1034] running on GCE node
I20260809 22:44:54.146330 598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:54.147332 598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:54.148556 598 hybrid_clock.cc:648] HybridClock initialized: now 1786315494148490 us; error 50 us; skew 500 ppm
I20260809 22:44:54.148856 598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:54 dist-test-slave-chsk krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:54.151937 598 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:44:54.153286 598 webserver.cc:460] Webserver started at http://127.21.27.130:38315/ using document root <none> and password file <none>
I20260809 22:44:54.153978 598 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:54.154085 598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:54.154372 598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:54.157199 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "df2a0575f44d4fd8929896a449cc86fe"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "24e1e1f1eb64a73f07e2772e842f9b28"
server_key_iv: "9ea2b26d4f4bf6df0602e7a04e6c8cbf"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.157928 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "df2a0575f44d4fd8929896a449cc86fe"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "24e1e1f1eb64a73f07e2772e842f9b28"
server_key_iv: "9ea2b26d4f4bf6df0602e7a04e6c8cbf"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.162741 598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 22:44:54.166028 617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.167465 598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:44:54.167645 598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "df2a0575f44d4fd8929896a449cc86fe"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "24e1e1f1eb64a73f07e2772e842f9b28"
server_key_iv: "9ea2b26d4f4bf6df0602e7a04e6c8cbf"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.167841 598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:54.185794 598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:54.189862 598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:54.190138 598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:54.191053 598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:54.192312 598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:54.192414 598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.192553 598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:54.192621 598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.224936 598 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:39725
I20260809 22:44:54.224979 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:39725 every 8 connection(s)
I20260809 22:44:54.226186 598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:44:54.231760 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 598
I20260809 22:44:54.231982 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:44:54.232602 21614 external_mini_cluster.cc:1428] Setting key 0ecbcbdbc14e8d152dc85d04ae05b102
Aug 09 22:44:54 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:54.246779 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.228578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43679 (local address 127.21.27.190:42869)
0809 22:44:54.228794 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.228800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.229887 (+ 1087us) server_negotiation.cc:407] Connection header received
0809 22:44:54.231638 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.231643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.231711 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.231797 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:54.233655 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.234553 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.236195 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.236602 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.236721 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.240132 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:54.240140 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:54.240148 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:54.240195 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:54.243798 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.244479 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.244487 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.244494 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.244655 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.245104 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.245108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.245110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.245395 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:54.245567 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.246171 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.246420 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:44:54.247699 739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.229053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42869 (local address 127.21.27.130:43679)
0809 22:44:54.229730 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.229771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.231349 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.232055 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.232066 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.232812 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:54.233503 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.233521 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.234701 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.234710 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.236057 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.236070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.236820 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.236828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.237033 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.237665 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:54.237691 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:54.239866 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:54.243928 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:54.243934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:54.243944 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:54.244348 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:54.244768 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:54.244770 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:54.244773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:54.244934 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:54.245525 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:54.245532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:54.245829 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.246077 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.246576 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":279,"threads_started":1}
I20260809 22:44:54.249085 737 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42869
I20260809 22:44:54.249398 737 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:54.250289 737 heartbeater.cc:511] Master 127.21.27.190:42869 requested a full tablet report, sending...
I20260809 22:44:54.251853 387 ts_manager.cc:194] Registered new tserver with Master: df2a0575f44d4fd8929896a449cc86fe (127.21.27.130:39725)
I20260809 22:44:54.252620 387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:43679
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:44:54.291895 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:44:54.618124 742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:44:54.618469 742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:44:54.618645 742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:44:54.626246 742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:44:54.626410 742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:44:54.626595 742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:44:54.637043 742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:44:54.638428 742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:44:54.639848 742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:44:54.653561 752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:54.653599 750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:44:54.654152 749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:44:54.654726 742 server_base.cc:1034] running on GCE node
I20260809 22:44:54.655445 742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:44:54.656436 742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:44:54.657637 742 hybrid_clock.cc:648] HybridClock initialized: now 1786315494657590 us; error 37 us; skew 500 ppm
I20260809 22:44:54.657930 742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:44:54 dist-test-slave-chsk krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:44:54.661197 742 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:44:54.662655 742 webserver.cc:460] Webserver started at http://127.21.27.131:39319/ using document root <none> and password file <none>
I20260809 22:44:54.663393 742 fs_manager.cc:359] Metadata directory not provided
I20260809 22:44:54.663498 742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:44:54.663765 742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:44:54.666692 742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "49a5638e505f44d1a9880198d29d4b01"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "f8e481acc2e02f45330209e5ffc156c6"
server_key_iv: "15aab31aeb351bd395bee1526e7c746a"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.667495 742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "49a5638e505f44d1a9880198d29d4b01"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "f8e481acc2e02f45330209e5ffc156c6"
server_key_iv: "15aab31aeb351bd395bee1526e7c746a"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.672801 742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:44:54.676553 759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.677919 742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:44:54.678089 742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "49a5638e505f44d1a9880198d29d4b01"
format_stamp: "Formatted at 2026-08-09 22:44:54 on dist-test-slave-chsk"
server_key: "f8e481acc2e02f45330209e5ffc156c6"
server_key_iv: "15aab31aeb351bd395bee1526e7c746a"
server_key_version: "encryptionkey@0"
I20260809 22:44:54.678303 742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:44:54.704449 742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:44:54.708830 742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:44:54.709149 742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:44:54.710119 742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:44:54.711550 742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:44:54.711648 742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.711828 742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:44:54.711884 742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:44:54.745363 742 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:34965
I20260809 22:44:54.745414 881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:34965 every 8 connection(s)
I20260809 22:44:54.746603 592 heartbeater.cc:503] Master 127.21.27.190:42869 was elected leader, sending a full tablet report...
I20260809 22:44:54.746654 742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:44:54.747289 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 742
I20260809 22:44:54.747536 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:44:54.748219 21614 external_mini_cluster.cc:1428] Setting key d2ceab86e8ca056f192823cfd5eb7cec
Aug 09 22:44:54 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:44:54.767163 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.749704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:55531 (local address 127.21.27.190:42869)
0809 22:44:54.750219 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.750225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.751185 (+ 960us) server_negotiation.cc:407] Connection header received
0809 22:44:54.753187 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.753192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.753276 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.753398 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:54.754861 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.755705 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.757212 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.757625 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.757756 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.761205 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:54.761211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:54.761216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:54.761253 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:54.764191 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.764892 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.764898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.764902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.765025 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.765444 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.765452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.765456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.765723 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:54.765873 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.766525 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.766740 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":275,"threads_started":1}
I20260809 22:44:54.768079 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.750334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42869 (local address 127.21.27.131:55531)
0809 22:44:54.751030 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.751069 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.752919 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.753642 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.753652 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.754162 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:44:54.754700 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.754717 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.755852 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.755858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.757056 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.757072 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.757844 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.757854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.758091 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.758763 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:44:54.758786 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:44:54.761005 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:44:54.764329 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:54.764337 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:54.764353 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:54.764768 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:54.765140 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:44:54.765145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:44:54.765149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:44:54.765335 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:44:54.765850 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:44:54.765858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:44:54.766174 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.766427 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.766944 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":287,"threads_started":1}
I20260809 22:44:54.769394 882 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42869
I20260809 22:44:54.769702 882 heartbeater.cc:464] Registering TS with master...
I20260809 22:44:54.770407 882 heartbeater.cc:511] Master 127.21.27.190:42869 requested a full tablet report, sending...
I20260809 22:44:54.771898 387 ts_manager.cc:194] Registered new tserver with Master: 49a5638e505f44d1a9880198d29d4b01 (127.21.27.131:34965)
I20260809 22:44:54.772689 387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:55531
I20260809 22:44:54.776216 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:44:54.793392 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.780746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.21.27.190:42869)
0809 22:44:54.780904 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.780909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.781448 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:44:54.781651 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.781656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.781726 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.781821 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:54.783380 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.784102 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.785326 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.785743 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.785883 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.788106 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:54.788112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:54.788117 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:54.788156 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:54.790587 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.791315 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.791322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.791326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.791457 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:54.791850 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:54.791855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:54.791858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:54.792185 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:54.792440 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.792787 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.793022 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:44:54.797387 387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:44:54.804204 387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:44:54.826545 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.815896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:34965 (local address 127.0.0.1:49154)
0809 22:44:54.816744 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.816796 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.816939 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.817972 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.817978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.818023 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:54.818507 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.818522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.820997 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.821005 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.823507 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.823525 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.825500 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.825508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.825717 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.825768 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.825903 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.826136 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":375,"threads_started":1}
I20260809 22:44:54.827484 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.816185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.21.27.131:34965)
0809 22:44:54.816922 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.816932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.816969 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:44:54.817162 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.817172 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.817447 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.817640 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:54.818752 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.820862 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.823680 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.824951 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.825075 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.825336 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.826935 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.827174 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":406,"threads_started":1}
I20260809 22:44:54.827589 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.818247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.21.27.130:39725)
0809 22:44:54.819796 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.819804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.819830 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:44:54.819928 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.819934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.820127 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.820323 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:54.820900 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.822834 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.825058 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.826454 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.826631 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.826822 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.827074 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.827263 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1268,"threads_started":1}
I20260809 22:44:54.827042 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.817126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39811 (local address 127.0.0.1:41290)
0809 22:44:54.817673 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.817693 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.817833 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.819908 (+ 2075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.819912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.819937 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:54.820405 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.820419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.822041 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.822047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.824310 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.824325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.826187 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.826192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.826380 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.826408 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.826481 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.826643 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":6,"thread_start_us":272,"threads_started":1}
I20260809 22:44:54.828011 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.817506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.21.27.129:39811)
0809 22:44:54.819330 (+ 1824us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.819338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.819365 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:44:54.819473 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.819484 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.819725 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.819919 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:54.820596 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.821869 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.824470 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.825961 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.827190 (+ 1229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.827415 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.827484 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.827668 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1613,"thread_start_us":1449,"threads_started":1}
I20260809 22:44:54.827600 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.817763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39725 (local address 127.0.0.1:51532)
0809 22:44:54.818510 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.818531 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.818671 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.820324 (+ 1653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.820334 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.820365 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:54.820752 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.820760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.822999 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.823006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.824907 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.824918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.826702 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.826708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.826881 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.826914 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.827022 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.827188 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":21,"thread_start_us":679,"threads_started":1}
I20260809 22:44:54.831720 522 tablet_service.cc:1467] Processing CreateTablet for tablet cd33661a81aa4802aff8c60309fb39e4 (DEFAULT_TABLE table=test-table [id=bbd100c0682843b1bcb82502d7733ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:54.831770 668 tablet_service.cc:1467] Processing CreateTablet for tablet cd33661a81aa4802aff8c60309fb39e4 (DEFAULT_TABLE table=test-table [id=bbd100c0682843b1bcb82502d7733ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:54.831770 811 tablet_service.cc:1467] Processing CreateTablet for tablet cd33661a81aa4802aff8c60309fb39e4 (DEFAULT_TABLE table=test-table [id=bbd100c0682843b1bcb82502d7733ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:44:54.832971 522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd33661a81aa4802aff8c60309fb39e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:54.833124 668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd33661a81aa4802aff8c60309fb39e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:54.833114 811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd33661a81aa4802aff8c60309fb39e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:44:54.844327 902 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Bootstrap starting.
I20260809 22:44:54.848554 903 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Bootstrap starting.
I20260809 22:44:54.849413 904 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Bootstrap starting.
I20260809 22:44:54.849472 902 tablet_bootstrap.cc:654] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:54.851032 902 log.cc:824] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:54.852866 904 tablet_bootstrap.cc:654] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:54.852883 903 tablet_bootstrap.cc:654] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Neither blocks nor log segments found. Creating new log.
I20260809 22:44:54.853950 903 log.cc:824] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:54.854276 904 log.cc:824] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Log is configured to *not* fsync() on all Append() calls
I20260809 22:44:54.854532 902 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: No bootstrap required, opened a new log
I20260809 22:44:54.854916 902 ts_tablet_manager.cc:1397] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 22:44:54.857125 903 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: No bootstrap required, opened a new log
I20260809 22:44:54.857160 904 tablet_bootstrap.cc:492] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: No bootstrap required, opened a new log
I20260809 22:44:54.857389 903 ts_tablet_manager.cc:1397] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 22:44:54.857416 904 ts_tablet_manager.cc:1397] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:44:54.864465 904 raft_consensus.cc:348] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.864841 904 raft_consensus.cc:374] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:54.864977 904 raft_consensus.cc:729] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a5638e505f44d1a9880198d29d4b01, State: Initialized, Role: FOLLOWER
I20260809 22:44:54.865585 904 consensus_queue.cc:260] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.865783 902 raft_consensus.cc:348] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.866315 902 raft_consensus.cc:374] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:54.866456 902 raft_consensus.cc:729] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3364ce619c040b7b8cf44b173f9bc82, State: Initialized, Role: FOLLOWER
I20260809 22:44:54.866711 882 heartbeater.cc:503] Master 127.21.27.190:42869 was elected leader, sending a full tablet report...
I20260809 22:44:54.867169 902 consensus_queue.cc:260] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.867448 904 ts_tablet_manager.cc:1428] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Time spent starting tablet: real 0.010s user 0.006s sys 0.006s
I20260809 22:44:54.867847 903 raft_consensus.cc:348] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.868477 903 raft_consensus.cc:374] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:44:54.868649 903 raft_consensus.cc:729] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2a0575f44d4fd8929896a449cc86fe, State: Initialized, Role: FOLLOWER
I20260809 22:44:54.869495 903 consensus_queue.cc:260] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.870814 737 heartbeater.cc:503] Master 127.21.27.190:42869 was elected leader, sending a full tablet report...
I20260809 22:44:54.872716 902 ts_tablet_manager.cc:1428] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
I20260809 22:44:54.873107 903 ts_tablet_manager.cc:1428] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260809 22:44:54.962199 593 tablet.cc:2369] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:44:54.978163 738 tablet.cc:2369] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:54.983793 910 raft_consensus.cc:482] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:44:54.984035 910 raft_consensus.cc:504] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.985746 910 leader_election.cc:288] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49a5638e505f44d1a9880198d29d4b01 (127.21.27.131:34965), f3364ce619c040b7b8cf44b173f9bc82 (127.21.27.129:39811)
I20260809 22:44:54.992832 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.986180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43319 (local address 127.21.27.131:34965)
0809 22:44:54.986363 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.986368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.986780 (+ 412us) server_negotiation.cc:407] Connection header received
0809 22:44:54.986989 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.986995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.987086 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.987268 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:54.987979 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.988928 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.990887 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.991791 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.991916 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.991967 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.992415 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.992552 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:44:54.992892 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.985994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:34965 (local address 127.21.27.130:43319)
0809 22:44:54.986671 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.986691 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.986838 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.987419 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.987422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.987444 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:54.987833 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.987848 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.989051 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.989057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.990735 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.990747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.992007 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.992015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.992233 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.992270 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.992366 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.992463 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":38,"thread_start_us":341,"threads_started":1}
I20260809 22:44:54.993001 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.986990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:56669 (local address 127.21.27.129:39811)
0809 22:44:54.987238 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:54.987246 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:54.987563 (+ 317us) server_negotiation.cc:407] Connection header received
0809 22:44:54.987711 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:54.987717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:54.987800 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:54.987906 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:44:54.988510 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.989344 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.991209 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.992049 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.992212 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.992281 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:54.992620 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 22:44:54.992746 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:44:54.993407 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:54.986752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39811 (local address 127.21.27.130:56669)
0809 22:44:54.987461 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 22:44:54.987474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 22:44:54.987586 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:44:54.987935 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:44:54.987939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:44:54.987964 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:44:54.988370 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.988383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.989491 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.989495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.991009 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:44:54.991023 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:44:54.992254 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:44:54.992258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:44:54.992443 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:54.992475 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:44:54.992567 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 22:44:54.992696 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":3,"thread_start_us":439,"threads_started":1}
I20260809 22:44:54.994064 834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd33661a81aa4802aff8c60309fb39e4" candidate_uuid: "df2a0575f44d4fd8929896a449cc86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a5638e505f44d1a9880198d29d4b01" is_pre_election: true
I20260809 22:44:54.994473 543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd33661a81aa4802aff8c60309fb39e4" candidate_uuid: "df2a0575f44d4fd8929896a449cc86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3364ce619c040b7b8cf44b173f9bc82" is_pre_election: true
I20260809 22:44:54.994709 834 raft_consensus.cc:2454] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2a0575f44d4fd8929896a449cc86fe in term 0.
I20260809 22:44:54.995034 543 raft_consensus.cc:2454] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df2a0575f44d4fd8929896a449cc86fe in term 0.
I20260809 22:44:54.995471 621 leader_election.cc:302] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a5638e505f44d1a9880198d29d4b01, df2a0575f44d4fd8929896a449cc86fe; no voters:
I20260809 22:44:54.995850 910 raft_consensus.cc:2781] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:44:54.995994 910 raft_consensus.cc:482] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:44:54.996129 910 raft_consensus.cc:3037] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:54.998286 910 raft_consensus.cc:504] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:54.998828 910 leader_election.cc:288] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [CANDIDATE]: Term 1 election: Requested vote from peers 49a5638e505f44d1a9880198d29d4b01 (127.21.27.131:34965), f3364ce619c040b7b8cf44b173f9bc82 (127.21.27.129:39811)
I20260809 22:44:54.999358 834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd33661a81aa4802aff8c60309fb39e4" candidate_uuid: "df2a0575f44d4fd8929896a449cc86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a5638e505f44d1a9880198d29d4b01"
I20260809 22:44:54.999454 543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd33661a81aa4802aff8c60309fb39e4" candidate_uuid: "df2a0575f44d4fd8929896a449cc86fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3364ce619c040b7b8cf44b173f9bc82"
I20260809 22:44:54.999596 834 raft_consensus.cc:3037] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:44:54.999661 543 raft_consensus.cc:3037] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:44:54.999706 883 tablet.cc:2369] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:44:55.002946 543 raft_consensus.cc:2454] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2a0575f44d4fd8929896a449cc86fe in term 1.
I20260809 22:44:55.002946 834 raft_consensus.cc:2454] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df2a0575f44d4fd8929896a449cc86fe in term 1.
I20260809 22:44:55.003453 621 leader_election.cc:302] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: df2a0575f44d4fd8929896a449cc86fe, f3364ce619c040b7b8cf44b173f9bc82; no voters:
I20260809 22:44:55.003793 910 raft_consensus.cc:2781] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:44:55.004288 910 raft_consensus.cc:686] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 1 LEADER]: Becoming Leader. State: Replica: df2a0575f44d4fd8929896a449cc86fe, State: Running, Role: LEADER
I20260809 22:44:55.004845 910 consensus_queue.cc:237] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } }
I20260809 22:44:55.009907 385 catalog_manager.cc:5495] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe reported cstate change: term changed from 0 to 1, leader changed from <none> to df2a0575f44d4fd8929896a449cc86fe (127.21.27.130). New cstate: current_term: 1 leader_uuid: "df2a0575f44d4fd8929896a449cc86fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df2a0575f44d4fd8929896a449cc86fe" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39725 } health_report { overall_health: HEALTHY } } }
I20260809 22:44:55.068369 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:55.063686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.21.27.130:39725)
0809 22:44:55.063873 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:55.063878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:55.063895 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:44:55.063981 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:55.063985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:55.064058 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:55.064187 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:44:55.064503 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.065237 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.066432 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.066811 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.066903 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:55.067370 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:44:55.067838 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:44:55.067952 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:55.068015 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 22:44:55.068077 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:44:55.076352 543 raft_consensus.cc:1261] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 1 FOLLOWER]: Refusing update from remote peer df2a0575f44d4fd8929896a449cc86fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:55.076391 834 raft_consensus.cc:1261] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 1 FOLLOWER]: Refusing update from remote peer df2a0575f44d4fd8929896a449cc86fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:44:55.077286 913 consensus_queue.cc:1035] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a5638e505f44d1a9880198d29d4b01" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 34965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:55.077669 910 consensus_queue.cc:1035] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3364ce619c040b7b8cf44b173f9bc82" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:44:55.115453 385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:55.115756 385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:44:55.119764 385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccd914fbfdbe4562a545df2e781b98a1: Sending DeleteTablet for 3 replicas of tablet cd33661a81aa4802aff8c60309fb39e4
I20260809 22:44:55.120998 811 tablet_service.cc:1514] Processing DeleteTablet for tablet cd33661a81aa4802aff8c60309fb39e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:55 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:49154
I20260809 22:44:55.121171 522 tablet_service.cc:1514] Processing DeleteTablet for tablet cd33661a81aa4802aff8c60309fb39e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:55 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:41290
I20260809 22:44:55.121585 668 tablet_service.cc:1514] Processing DeleteTablet for tablet cd33661a81aa4802aff8c60309fb39e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:44:55 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:51532
I20260809 22:44:55.126470 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 455
I20260809 22:44:55.136312 927 tablet_replica.cc:330] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: stopping tablet replica
I20260809 22:44:55.136988 927 raft_consensus.cc:2229] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:55.137661 927 raft_consensus.cc:2258] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:55.140112 927 ts_tablet_manager.cc:1905] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:55.147835 927 ts_tablet_manager.cc:1918] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:55.148098 927 log.cc:1196] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/cd33661a81aa4802aff8c60309fb39e4
I20260809 22:44:55.148763 927 ts_tablet_manager.cc:1939] T cd33661a81aa4802aff8c60309fb39e4 P 49a5638e505f44d1a9880198d29d4b01: Deleting consensus metadata
I20260809 22:44:55.150811 373 catalog_manager.cc:4841] TS 49a5638e505f44d1a9880198d29d4b01 (127.21.27.131:34965): tablet cd33661a81aa4802aff8c60309fb39e4 (table test-table [id=bbd100c0682843b1bcb82502d7733ddf]) successfully deleted
I20260809 22:44:55.151813 929 tablet_replica.cc:330] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: stopping tablet replica
I20260809 22:44:55.152413 929 raft_consensus.cc:2229] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:44:55.153292 929 raft_consensus.cc:2258] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:55.155391 928 tablet_replica.cc:330] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: stopping tablet replica
I20260809 22:44:55.155853 928 raft_consensus.cc:2229] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:44:55.156399 928 raft_consensus.cc:2258] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:44:55.158680 928 ts_tablet_manager.cc:1905] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:44:55.159003 929 ts_tablet_manager.cc:1905] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:44:55.166085 928 ts_tablet_manager.cc:1918] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:55.166337 928 log.cc:1196] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/cd33661a81aa4802aff8c60309fb39e4
I20260809 22:44:55.166396 929 ts_tablet_manager.cc:1918] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:44:55.166649 929 log.cc:1196] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/cd33661a81aa4802aff8c60309fb39e4
I20260809 22:44:55.167018 928 ts_tablet_manager.cc:1939] T cd33661a81aa4802aff8c60309fb39e4 P f3364ce619c040b7b8cf44b173f9bc82: Deleting consensus metadata
I20260809 22:44:55.167407 929 ts_tablet_manager.cc:1939] T cd33661a81aa4802aff8c60309fb39e4 P df2a0575f44d4fd8929896a449cc86fe: Deleting consensus metadata
I20260809 22:44:55.169147 373 catalog_manager.cc:4841] TS f3364ce619c040b7b8cf44b173f9bc82 (127.21.27.129:39811): tablet cd33661a81aa4802aff8c60309fb39e4 (table test-table [id=bbd100c0682843b1bcb82502d7733ddf]) successfully deleted
I20260809 22:44:55.169471 373 catalog_manager.cc:4841] TS df2a0575f44d4fd8929896a449cc86fe (127.21.27.130:39725): tablet cd33661a81aa4802aff8c60309fb39e4 (table test-table [id=bbd100c0682843b1bcb82502d7733ddf]) successfully deleted
I20260809 22:44:55.172772 899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:55.127930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.21.27.129:39811)
0809 22:44:55.128173 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:55.128182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:55.148112 (+ 19930us) server_negotiation.cc:407] Connection header received
0809 22:44:55.148778 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:55.148788 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:55.148902 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:55.149008 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:55.153961 (+ 4953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.155494 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.157362 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.158047 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.158192 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:55.163910 (+ 5718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:55.163928 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:55.163945 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:55.163997 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:55.169302 (+ 5305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.170428 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.170435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.170438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.170571 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.171042 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.171045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.171047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.171562 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:55.171807 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:55.172247 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 22:44:55.172414 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:44:55.173481 587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:55.318666 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 455
I20260809 22:44:55.339032 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 598
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:44:55.357076 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:55.339926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.21.27.130:39725)
0809 22:44:55.340115 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:55.340120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:55.340179 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:44:55.340387 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:55.340391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:55.340453 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:55.340632 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:55.342307 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.343126 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.344955 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.345385 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.345481 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:55.350693 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:55.350708 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:55.350723 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:55.350769 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:55.354048 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.354796 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.354805 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.354807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.354955 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.355396 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.355408 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.355412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.355795 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:55.356172 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:55.356457 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 22:44:55.356695 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:44:55.357772 732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:55.473203 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 598
I20260809 22:44:55.495950 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 742
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:44:55.510612 933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:44:55.496796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.21.27.131:34965)
0809 22:44:55.497262 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 22:44:55.497267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:44:55.497283 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:44:55.497338 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:44:55.497342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:44:55.497413 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:44:55.497503 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:44:55.498698 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.499487 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.500686 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:44:55.501063 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:44:55.501148 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:44:55.505411 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:44:55.505425 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:44:55.505442 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:44:55.505490 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:44:55.508228 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.508816 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.508823 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.508825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.508959 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:44:55.509259 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:44:55.509263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:44:55.509264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:44:55.509595 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:44:55.509813 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 22:44:55.510058 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 22:44:55.510215 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":248,"threads_started":1}
I20260809 22:44:55.511266 877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:55.601498 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 742
I20260809 22:44:55.619923 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 354
I20260809 22:44:55.621122 419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:44:55.733510 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 354
2026-08-09T22:44:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5132 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[939](info): set up 2 sockets
Aug 09 22:44:55 dist-test-slave-chsk krb5kdc[939](info): commencing operation
krb5kdc: starting...
W20260809 22:44:56.912839 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:44:57 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:44:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangeradmin_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangeradmin_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangerlookup_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangerlookup_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
I20260809 22:44:57.288417 21614 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvPTdCg/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/postgres -l logfile start
2026-08-09 22:44:59.897 UTC [975] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:44:59.897 UTC [975] LOG: listening on IPv4 address "127.21.27.148", port 57345
2026-08-09 22:44:59.900 UTC [975] LOG: listening on Unix socket "/tmp/.s.PGSQL.57345"
2026-08-09 22:44:59.925 UTC [978] LOG: database system was shut down at 2026-08-09 22:44:58 UTC
2026-08-09 22:44:59.928 UTC [975] LOG: database system is ready to accept connections
I20260809 22:45:00.964365 21614 mini_postgres.cc:96] Postgres bound to 57345
2026-08-09 22:45:00.989 UTC [988] FATAL: database "slave" does not exist
127.21.27.148:57345 - accepting connections
I20260809 22:45:00.990290 21614 mini_ranger.cc:161] Starting Ranger...
I20260809 22:45:01.024627 21614 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:45:01.513384 21614 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:45:01.513504 21614 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin
2026-08-09 22:45:01,915 [I] DB FLAVOR :POSTGRES
2026-08-09 22:45:01,915 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:45:01,915 [I] Checking connection..
2026-08-09 22:45:01,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:45:02,192 [I] Checking connection passed.
2026-08-09 22:45:02,192 [I] --------- Verifying version history table ---------
2026-08-09 22:45:02,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:02,463 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:45:02,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:02,734 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:45:02,735 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:45:02,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:45:02.994 UTC [1059] WARNING: there is no transaction in progress
2026-08-09 22:45:03,002 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:45:03,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:03,273 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:45:03,273 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:45:03,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:45:03,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:45:03,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-chsk.c.gcp-upstream.internal','N') ;"
2026-08-09 22:45:04,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:45:04,290 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:45:04,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:45:04,555 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:45:04,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:45:04,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:45:05,076 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:45:05,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:45:06.053 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.068 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.083 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.098 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.114 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.132 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.197 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.203 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.217 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.227 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.236 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.250 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.259 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.266 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.273 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06.280 UTC [1177] WARNING: there is no transaction in progress
2026-08-09 22:45:06,920 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:45:06,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:45:07,158 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:45:07,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:45:07,427 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:45:07,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:45:07,692 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:45:07,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:45:07,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:45:08,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-chsk.c.gcp-upstream.internal';"
2026-08-09 22:45:08,422 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:45:08,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:45:08,681 [I] DB_PATCHES have already been applied
I20260809 22:45:08.687403 21614 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvPTdCg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:45:08.687529 21614 mini_ranger.cc:193] Using host: 127.21.27.148
I20260809 22:45:08.707237 21614 mini_ranger.cc:227] Ranger admin URL: http://127.21.27.148:54255
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:45:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:45:09.938092 21614 mini_ranger.cc:160] Time spent starting Ranger: real 8.948s user 0.003s sys 0.056s
Aug 09, 2026 10:45:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:45:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 10:45:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:45:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:45:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:45:32.284413 21614 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangerkms_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/rangerkms_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.27.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.27.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:45:32 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315532, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:45:32.477916 21614 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:45:32.504151 21614 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:45:33.421972 21614 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:45:33.422202 21614 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms
I20260809 22:45:33.422325 21614 mini_ranger_kms.cc:227] Using postgres at 127.21.27.148:57345
2026-08-09 22:45:34,113 [I] DB FLAVOR :POSTGRES
2026-08-09 22:45:34,113 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:45:34,114 [I] Checking connection
2026-08-09 22:45:34,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:45:34,383 [I] connection success
2026-08-09 22:45:34,383 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:45:34,383 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:45:34,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:45:34,618 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:45:34,619 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:45:34,619 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:45:34,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:57345/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:45:34,884 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:45:35.420588 21614 mini_ranger_kms.cc:310] Created kms service
I20260809 22:45:35.631042 21614 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:45:35.682843 21614 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:45:35 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315532, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:36.152572 21614 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:45:36.153885 21614 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:45:36.153967 21614 mini_ranger_kms.cc:240] Using host: 127.21.27.148
I20260809 22:45:36.177857 21614 mini_ranger_kms.cc:276] Ranger KMS PID: 1385
I20260809 22:45:36.178081 21614 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.27.148:42255
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:45:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:45:38 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315538, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:38.645139 21614 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.167s user 0.112s sys 0.432s
I20260809 22:45:38.645514 21614 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:45:38.645768 21614 mini_ranger_kms.cc:401] 127.21.27.148:42255/kms/v1/keys
Aug 09 22:45:38 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315538, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
Aug 09 22:45:39 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315538, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
Aug 09 22:45:41 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:45:41.988672 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39415
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:39415
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:42255/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:42.637004 1442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:42.637549 1442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:42.637800 1442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:42.648125 1442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:45:42.648341 1442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:42.648497 1442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:42.648677 1442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:45:42.648828 1442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:45:42.662889 1442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.27.148:42255/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:39415
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:45:42.664611 1442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:42.666234 1442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:42.688987 1448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:42.689116 1451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:42.689098 1449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:42.690590 1442 server_base.cc:1034] running on GCE node
I20260809 22:45:42.691581 1442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:42.693184 1442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:42.694451 1442 hybrid_clock.cc:648] HybridClock initialized: now 1786315542694406 us; error 39 us; skew 500 ppm
I20260809 22:45:42.694872 1442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:42 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:42.698472 1442 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:45:42.700665 1442 webserver.cc:460] Webserver started at http://127.21.27.190:35347/ using document root <none> and password file <none>
I20260809 22:45:42.701380 1442 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:42.701520 1442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:42.701876 1442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:45:42 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315542, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:42.770241 1442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "bae2c6656db24d08b1d495e240c087ac"
format_stamp: "Formatted at 2026-08-09 22:45:42 on dist-test-slave-chsk"
server_key: "9a111e257df0de7f528750c4b15a696c"
server_key_iv: "b42403aa8f6aa0f93319a79b3f8bb07f"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:42.771605 1442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "bae2c6656db24d08b1d495e240c087ac"
format_stamp: "Formatted at 2026-08-09 22:45:42 on dist-test-slave-chsk"
server_key: "9a111e257df0de7f528750c4b15a696c"
server_key_iv: "b42403aa8f6aa0f93319a79b3f8bb07f"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:42.779402 1442 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.006s
I20260809 22:45:42.802605 1458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:42.804308 1442 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:45:42.804566 1442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "bae2c6656db24d08b1d495e240c087ac"
format_stamp: "Formatted at 2026-08-09 22:45:42 on dist-test-slave-chsk"
server_key: "9a111e257df0de7f528750c4b15a696c"
server_key_iv: "b42403aa8f6aa0f93319a79b3f8bb07f"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:42.804823 1442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:42.830864 1442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:42.836401 1442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:42.836995 1442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:42.885105 1442 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:39415
I20260809 22:45:42.885178 1515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:39415 every 8 connection(s)
I20260809 22:45:42.887077 1442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:45:42.892676 1516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:42.893597 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 1442
I20260809 22:45:42.893826 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:45:42.894877 21614 external_mini_cluster.cc:1428] Setting key b03b340f57daf45578ad7aee9b704346
I20260809 22:45:42.912775 1516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: Bootstrap starting.
Aug 09 22:45:42 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:45:42.918291 1516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:42.920514 1516 log.cc:824] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:42.925443 1516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: No bootstrap required, opened a new log
I20260809 22:45:42.926148 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:42.898079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.21.27.190:39415)
0809 22:45:42.899721 (+ 1642us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:42.899736 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:42.899790 (+ 54us) server_negotiation.cc:407] Connection header received
0809 22:45:42.903013 (+ 3223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:42.903046 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:42.903727 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:42.904252 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:42.905949 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.907893 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.909375 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:42.910441 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:42.910704 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:42.916205 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:42.916215 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:42.916234 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:42.916285 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:42.921055 (+ 4770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.921966 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.921975 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.921981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.922282 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:42.922790 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:42.922796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:42.922800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:42.923514 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:42.923869 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:42.924399 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 22:45:42.924857 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"thread_start_us":924,"threads_started":1}
I20260809 22:45:42.939301 1516 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } }
I20260809 22:45:42.939895 1516 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:42.940057 1516 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae2c6656db24d08b1d495e240c087ac, State: Initialized, Role: FOLLOWER
I20260809 22:45:42.940900 1516 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } }
I20260809 22:45:42.941243 1516 raft_consensus.cc:388] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:45:42.941560 1516 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:45:42.941777 1516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:42.945853 1516 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } }
I20260809 22:45:42.946590 1516 leader_election.cc:302] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bae2c6656db24d08b1d495e240c087ac; no voters:
I20260809 22:45:42.947768 1516 leader_election.cc:288] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:45:42.947947 1524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:42.949622 1524 raft_consensus.cc:686] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [term 1 LEADER]: Becoming Leader. State: Replica: bae2c6656db24d08b1d495e240c087ac, State: Running, Role: LEADER
I20260809 22:45:42.950498 1524 consensus_queue.cc:237] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } }
I20260809 22:45:42.951495 1516 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:45:42.957223 1525 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bae2c6656db24d08b1d495e240c087ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } } }
I20260809 22:45:42.958640 1525 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:42.958072 1526 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae2c6656db24d08b1d495e240c087ac. Latest consensus state: current_term: 1 leader_uuid: "bae2c6656db24d08b1d495e240c087ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae2c6656db24d08b1d495e240c087ac" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 39415 } } }
I20260809 22:45:42.958963 1526 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac [sys.catalog]: This master's current role is: LEADER
I20260809 22:45:42.962934 1533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:45:42.972419 1533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:45:42.973811 1516 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:45:43.013368 1533 catalog_manager.cc:1347] Generated new cluster ID: bfc52b25843241589609af80264ceb6b
I20260809 22:45:43.013703 1533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:45:43.045503 1533 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:45:43.048139 1533 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:45:43.068424 1533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: Generated new TSK 0
I20260809 22:45:43.069823 1533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:45:43.935880 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:42255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:45:44 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:45:44.552394 1569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:44.552950 1569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:44.553272 1569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:44.563357 1569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:44.563534 1569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:44.563769 1569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:45:44.577194 1569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.27.148:42255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:45:44.578756 1569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:44.580435 1569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:44.606913 1581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.607033 1580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:44.608572 1583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:44.612301 1569 server_base.cc:1034] running on GCE node
I20260809 22:45:44.613912 1569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:44.626418 1569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:44.627918 1569 hybrid_clock.cc:648] HybridClock initialized: now 1786315544627704 us; error 123 us; skew 500 ppm
I20260809 22:45:44.629257 1569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:44 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:44.634490 1569 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:45:44.641909 1569 webserver.cc:460] Webserver started at http://127.21.27.129:40347/ using document root <none> and password file <none>
I20260809 22:45:44.643177 1569 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:44.643440 1569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:44.643909 1569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:45:44 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:44.666250 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-chsk"
server_key: "a9e44629cedeb1f58ecf3dd724d4ed9f"
server_key_iv: "204a6337e9e70eeb5d0d595fe8b73add"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:44.667816 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-chsk"
server_key: "a9e44629cedeb1f58ecf3dd724d4ed9f"
server_key_iv: "204a6337e9e70eeb5d0d595fe8b73add"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:44.700546 1569 fs_manager.cc:691] Time spent creating directory manager: real 0.032s user 0.016s sys 0.005s
I20260809 22:45:44.732769 1592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:44.735045 1569 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.004s
I20260809 22:45:44.735580 1569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe"
format_stamp: "Formatted at 2026-08-09 22:45:44 on dist-test-slave-chsk"
server_key: "a9e44629cedeb1f58ecf3dd724d4ed9f"
server_key_iv: "204a6337e9e70eeb5d0d595fe8b73add"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:44.736024 1569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:44.783417 1569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:44.791019 1569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:44.791790 1569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:44.794004 1569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:44.796640 1569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:44.796868 1569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:45:44.797109 1569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:44.797252 1569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:45:44 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:45.057617 1569 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:42467
I20260809 22:45:45.057675 1718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:42467 every 8 connection(s)
I20260809 22:45:45.059613 1569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:45:45.062832 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 1569
I20260809 22:45:45.063057 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:45:45.064069 21614 external_mini_cluster.cc:1428] Setting key 83ce6c03e4f49bdfa4e517fd0efec7b5
Aug 09 22:45:45 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
I20260809 22:45:45.112303 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.070661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:52753 (local address 127.21.27.190:39415)
0809 22:45:45.080496 (+ 9835us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:45.080503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:45.081246 (+ 743us) server_negotiation.cc:407] Connection header received
0809 22:45:45.083907 (+ 2661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:45.083914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:45.084024 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:45.084138 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:45.085983 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.087607 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.090814 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.091772 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.091888 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.101956 (+ 10068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:45.101967 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:45.101973 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:45.102052 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:45.107376 (+ 5324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.108315 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.108321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.108325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.108469 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:45.109086 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:45.109092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:45.109095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:45.109610 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:45.109753 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:45.111319 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0809 22:45:45.111652 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9693,"thread_start_us":10290,"threads_started":1}
I20260809 22:45:45.115228 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:45.069847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39415 (local address 127.21.27.129:52753)
0809 22:45:45.080895 (+ 11048us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:45.080950 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:45.083444 (+ 2494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:45.084670 (+ 1226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:45.084685 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:45.085231 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:45.085793 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.085810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.087846 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.087854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.090626 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:45.090645 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:45.092391 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:45.092401 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:45.092929 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:45.097123 (+ 4194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:45.097176 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:45.101407 (+ 4231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:45.107579 (+ 6172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.107588 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.107603 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.108163 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.108613 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:45.108622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:45.108625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:45.108906 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:45.109849 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:45.109859 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:45.110401 (+ 542us) client_negotiation.cc:769] Sending connection context
0809 22:45:45.110774 (+ 373us) client_negotiation.cc:240] Negotiation successful
0809 22:45:45.113149 (+ 2375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10740,"thread_start_us":9953,"threads_started":1}
I20260809 22:45:45.120925 1719 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39415
I20260809 22:45:45.121302 1719 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:45.122079 1719 heartbeater.cc:511] Master 127.21.27.190:39415 requested a full tablet report, sending...
I20260809 22:45:45.124848 1479 ts_manager.cc:194] Registered new tserver with Master: 10a5a586b00b4c87bcf0eeb92ac9ecfe (127.21.27.129:42467)
I20260809 22:45:45.127545 1479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:52753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:45:45.152932 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:42255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:45:45 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315544, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
W20260809 22:45:45.885455 1726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:45.885905 1726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:45.886144 1726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:45.894023 1726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:45.894191 1726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:45.894392 1726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:45:45.907285 1726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.27.148:42255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:45:45.909006 1726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:45.910663 1726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:45.933611 1743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:45.935490 1726 server_base.cc:1034] running on GCE node
W20260809 22:45:45.935698 1745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:45.935859 1742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:45.936802 1726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:45.938179 1726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:45.939499 1726 hybrid_clock.cc:648] HybridClock initialized: now 1786315545939332 us; error 88 us; skew 500 ppm
I20260809 22:45:45.940021 1726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:45 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:45.945863 1726 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:45:45.948498 1726 webserver.cc:460] Webserver started at http://127.21.27.130:40799/ using document root <none> and password file <none>
I20260809 22:45:45.949648 1726 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:45.949889 1726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:45.950415 1726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:45:45 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:45.970604 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "d0d681f4802244e8bf7e524becbde861"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-chsk"
server_key: "63416caf3808c22ff139c3dcc0dd7fea"
server_key_iv: "238c2ae7c6b083d9c354ff4515512195"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:45.972025 1726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "d0d681f4802244e8bf7e524becbde861"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-chsk"
server_key: "63416caf3808c22ff139c3dcc0dd7fea"
server_key_iv: "238c2ae7c6b083d9c354ff4515512195"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:45.980321 1726 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.002s
I20260809 22:45:45.998672 1753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.000273 1726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:45:46.000521 1726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "d0d681f4802244e8bf7e524becbde861"
format_stamp: "Formatted at 2026-08-09 22:45:45 on dist-test-slave-chsk"
server_key: "63416caf3808c22ff139c3dcc0dd7fea"
server_key_iv: "238c2ae7c6b083d9c354ff4515512195"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:46.000845 1726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:46.033833 1726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:46.038587 1726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:46.039069 1726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:46.040613 1726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:46.042291 1726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:46.042580 1726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:45:46.042778 1726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:46.042917 1726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.121757 1726 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:35019
I20260809 22:45:46.121870 1882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:35019 every 8 connection(s)
I20260809 22:45:46.123400 1726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:45:46.131073 1719 heartbeater.cc:503] Master 127.21.27.190:39415 was elected leader, sending a full tablet report...
I20260809 22:45:46.132784 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 1726
I20260809 22:45:46.132959 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:45:46.133685 21614 external_mini_cluster.cc:1428] Setting key 496b46851222e805db13e9f6eaf755c0
Aug 09 22:45:46 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315545, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:45:46.152920 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.126490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:51215 (local address 127.21.27.190:39415)
0809 22:45:46.127318 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:46.127333 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:46.128620 (+ 1287us) server_negotiation.cc:407] Connection header received
0809 22:45:46.130967 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:46.130973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:46.131096 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:46.131386 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:46.133842 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.135075 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.138237 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.138740 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.138845 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.144601 (+ 5756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:46.144608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:46.144612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:46.144667 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:46.148738 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.149744 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.149750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.149754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.149949 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:46.150503 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:46.150508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:46.150522 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:46.150887 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:46.151112 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:46.151992 (+ 880us) server_negotiation.cc:299] Negotiation successful
0809 22:45:46.152286 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":457,"threads_started":1}
I20260809 22:45:46.153993 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:46.127391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39415 (local address 127.21.27.130:51215)
0809 22:45:46.128446 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:46.128493 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:46.130703 (+ 2210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:46.131649 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:46.131660 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:46.132465 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:46.133612 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.133636 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.135279 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.135286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.138047 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:46.138084 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:46.139073 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:46.139084 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:46.139540 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:46.140473 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:46.140507 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:46.144304 (+ 3797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:46.148929 (+ 4625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.148941 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.148969 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.149587 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.150092 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:46.150097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:46.150100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:46.150375 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:46.151084 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:46.151093 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:46.151544 (+ 451us) client_negotiation.cc:769] Sending connection context
0809 22:45:46.151845 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 22:45:46.152525 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":503,"threads_started":1}
I20260809 22:45:46.155946 1883 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39415
I20260809 22:45:46.156373 1883 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:46.157544 1883 heartbeater.cc:511] Master 127.21.27.190:39415 requested a full tablet report, sending...
I20260809 22:45:46.159708 1479 ts_manager.cc:194] Registered new tserver with Master: d0d681f4802244e8bf7e524becbde861 (127.21.27.130:35019)
I20260809 22:45:46.160645 1479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:51215
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:45:46.202715 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:42255/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:45:46.775540 1889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:45:46.775853 1889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:45:46.776003 1889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:45:46.785025 1889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:45:46.785254 1889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:45:46.785456 1889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:45:46.797684 1889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.27.148:42255/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:39415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:45:46.798913 1889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:45:46.800256 1889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:45:46.817440 1894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:46.817674 1896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:45:46.817886 1898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:45:46.819473 1889 server_base.cc:1034] running on GCE node
I20260809 22:45:46.820204 1889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:45:46.821309 1889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:45:46.822562 1889 hybrid_clock.cc:648] HybridClock initialized: now 1786315546822437 us; error 72 us; skew 500 ppm
I20260809 22:45:46.822956 1889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:45:46 dist-test-slave-chsk krb5kdc[939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:45:46.827390 1889 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:45:46.829164 1889 webserver.cc:460] Webserver started at http://127.21.27.131:36983/ using document root <none> and password file <none>
I20260809 22:45:46.829954 1889 fs_manager.cc:359] Metadata directory not provided
I20260809 22:45:46.830163 1889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:45:46.830595 1889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:45:46 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:45:46.845260 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "8e8b523947a948e786ad2129591880d9"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-chsk"
server_key: "35d0cc8240ca258b3f630fe0998da39c"
server_key_iv: "032cdf4766a59f7959259015b7b1cb62"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:46.846261 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "8e8b523947a948e786ad2129591880d9"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-chsk"
server_key: "35d0cc8240ca258b3f630fe0998da39c"
server_key_iv: "032cdf4766a59f7959259015b7b1cb62"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:46.852623 1889 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 22:45:46.866622 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.868177 1889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 22:45:46.868440 1889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "8e8b523947a948e786ad2129591880d9"
format_stamp: "Formatted at 2026-08-09 22:45:46 on dist-test-slave-chsk"
server_key: "35d0cc8240ca258b3f630fe0998da39c"
server_key_iv: "032cdf4766a59f7959259015b7b1cb62"
server_key_version: "kuduclusterkey@0"
I20260809 22:45:46.868731 1889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:45:46.915333 1889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:45:46.920596 1889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:45:46.921139 1889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:45:46.922551 1889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:45:46.924696 1889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:45:46.924817 1889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:46.924942 1889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:45:46.924991 1889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:45:47.009882 1889 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:43167
I20260809 22:45:47.009876 2037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:43167 every 8 connection(s)
I20260809 22:45:47.011708 1889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:45:47.019335 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 1889
I20260809 22:45:47.019528 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:45:47.020375 21614 external_mini_cluster.cc:1428] Setting key 1ffae6a86ae00fa1154925cab3a789b6
Aug 09 22:45:47 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315546, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:45:47.038573 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.014766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:38035 (local address 127.21.27.190:39415)
0809 22:45:47.015438 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.015442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.016796 (+ 1354us) server_negotiation.cc:407] Connection header received
0809 22:45:47.019390 (+ 2594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.019395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.019507 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.019702 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.022005 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.023542 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.025714 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.026264 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.026361 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.030601 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.030607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.030610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.030667 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.034438 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.035559 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.035565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.035568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.035776 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.036511 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.036515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.036518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.037033 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.037174 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.037978 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.038182 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":466,"threads_started":1}
I20260809 22:45:47.039641 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.015426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:39415 (local address 127.21.27.131:38035)
0809 22:45:47.016553 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.016597 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.019054 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.020019 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.020031 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.020946 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:45:47.021796 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.021813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.023698 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.023706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.025552 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.025564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.026497 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.026502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.026737 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.027488 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:45:47.027511 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:45:47.030336 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:45:47.034594 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:47.034601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:47.034615 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:47.035413 (+ 798us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:47.035912 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:45:47.035919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:45:47.035924 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:45:47.036372 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:45:47.037197 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:45:47.037206 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:45:47.037605 (+ 399us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.037848 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.038335 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":582,"threads_started":1}
I20260809 22:45:47.040984 2039 heartbeater.cc:347] Connected to a master server at 127.21.27.190:39415
I20260809 22:45:47.041307 2039 heartbeater.cc:464] Registering TS with master...
I20260809 22:45:47.042027 2039 heartbeater.cc:511] Master 127.21.27.190:39415 requested a full tablet report, sending...
I20260809 22:45:47.043666 1479 ts_manager.cc:194] Registered new tserver with Master: 8e8b523947a948e786ad2129591880d9 (127.21.27.131:43167)
I20260809 22:45:47.044716 1479 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:38035
I20260809 22:45:47.049567 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:45:47.072849 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.055756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50318 (local address 127.21.27.190:39415)
0809 22:45:47.055940 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.055944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.056895 (+ 951us) server_negotiation.cc:407] Connection header received
0809 22:45:47.057110 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.057115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.057205 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.057315 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.059189 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.060063 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.061769 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.062452 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.062596 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.065766 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.065771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.065775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.065809 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.069337 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.070208 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.070215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.070219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.070405 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.070893 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.070913 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.070917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.071336 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.071754 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.072083 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.072352 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:45:47.077551 1479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:45:47.088095 1479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:45:47.126520 2053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.106325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43167 (local address 127.0.0.1:35814)
0809 22:45:47.110105 (+ 3780us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.110160 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.110596 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.112821 (+ 2225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.112834 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.112883 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.114032 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.114053 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.117541 (+ 3488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.117547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.122104 (+ 4557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.122120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.125055 (+ 2935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.125063 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.125354 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.125442 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.125550 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.125754 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3467,"mutex_wait_us":35,"thread_start_us":2101,"threads_started":1}
I20260809 22:45:47.127358 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.107411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.21.27.131:43167)
0809 22:45:47.109677 (+ 2266us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.109782 (+ 105us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.111450 (+ 1668us) server_negotiation.cc:407] Connection header received
0809 22:45:47.111585 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.111596 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.111915 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.114376 (+ 2461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.115006 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.117354 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.122391 (+ 5037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.124664 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.125991 (+ 1327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.126366 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.126474 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.126734 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1960,"thread_start_us":1605,"threads_started":1}
I20260809 22:45:47.128638 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.105624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.21.27.130:35019)
0809 22:45:47.107649 (+ 2025us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.107668 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.110315 (+ 2647us) server_negotiation.cc:407] Connection header received
0809 22:45:47.110965 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.110979 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.111509 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.111878 (+ 369us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.115312 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.117798 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.122369 (+ 4571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.125030 (+ 2661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.125183 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.125529 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.127872 (+ 2343us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.128092 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":1209,"threads_started":1}
I20260809 22:45:47.129830 2055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.105384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:35019 (local address 127.0.0.1:47738)
0809 22:45:47.110105 (+ 4721us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.110158 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.110591 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.113742 (+ 3151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.113753 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.113791 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.114936 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.114954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.118096 (+ 3142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.118106 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.122148 (+ 4042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.122166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.127111 (+ 4945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.127120 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.127406 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.127440 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.127518 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.127668 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4242,"mutex_wait_us":30,"thread_start_us":1685,"threads_started":1}
I20260809 22:45:47.132897 1964 tablet_service.cc:1467] Processing CreateTablet for tablet 5abc25e732314858a0920cb704b40e99 (DEFAULT_TABLE table=test-table [id=8bfdf70ba0684468af94b3301f16432e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:47.135036 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.108328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:42467 (local address 127.0.0.1:44786)
0809 22:45:47.119747 (+ 11419us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.119795 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.120082 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.121420 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.121425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.121452 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.123973 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.123987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.126736 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.126741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.131653 (+ 4912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.131666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.133899 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.133909 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.134178 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.134210 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.134315 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.134460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11192,"thread_start_us":10584,"threads_started":1}
I20260809 22:45:47.134775 1807 tablet_service.cc:1467] Processing CreateTablet for tablet 5abc25e732314858a0920cb704b40e99 (DEFAULT_TABLE table=test-table [id=8bfdf70ba0684468af94b3301f16432e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:47.135599 1964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abc25e732314858a0920cb704b40e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.136090 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.108984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.21.27.129:42467)
0809 22:45:47.117118 (+ 8134us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.117128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.120239 (+ 3111us) server_negotiation.cc:407] Connection header received
0809 22:45:47.120411 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.120420 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.120753 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.121104 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.124191 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.126484 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.131819 (+ 5335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.133037 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.133226 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.133561 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.135468 (+ 1907us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.135694 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7877,"thread_start_us":7557,"threads_started":1}
I20260809 22:45:47.136750 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abc25e732314858a0920cb704b40e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.141727 1646 tablet_service.cc:1467] Processing CreateTablet for tablet 5abc25e732314858a0920cb704b40e99 (DEFAULT_TABLE table=test-table [id=8bfdf70ba0684468af94b3301f16432e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:45:47.149317 1646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abc25e732314858a0920cb704b40e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:45:47.157783 2060 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Bootstrap starting.
I20260809 22:45:47.162845 2061 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Bootstrap starting.
I20260809 22:45:47.164987 1883 heartbeater.cc:503] Master 127.21.27.190:39415 was elected leader, sending a full tablet report...
I20260809 22:45:47.166631 2060 tablet_bootstrap.cc:654] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.168787 2061 tablet_bootstrap.cc:654] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.169293 2060 log.cc:824] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:47.171119 2061 log.cc:824] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:47.175046 2060 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: No bootstrap required, opened a new log
I20260809 22:45:47.175688 2060 ts_tablet_manager.cc:1397] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.006s
I20260809 22:45:47.175922 2061 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: No bootstrap required, opened a new log
I20260809 22:45:47.176110 2065 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Bootstrap starting.
I20260809 22:45:47.176393 2061 ts_tablet_manager.cc:1397] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260809 22:45:47.181651 2065 tablet_bootstrap.cc:654] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Neither blocks nor log segments found. Creating new log.
I20260809 22:45:47.183386 2065 log.cc:824] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Log is configured to *not* fsync() on all Append() calls
I20260809 22:45:47.187037 2065 tablet_bootstrap.cc:492] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: No bootstrap required, opened a new log
I20260809 22:45:47.187433 2065 ts_tablet_manager.cc:1397] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260809 22:45:47.192730 2060 raft_consensus.cc:348] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.192703 2061 raft_consensus.cc:348] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.193326 2061 raft_consensus.cc:374] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.193388 2060 raft_consensus.cc:374] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.193550 2061 raft_consensus.cc:729] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d681f4802244e8bf7e524becbde861, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.193547 2060 raft_consensus.cc:729] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8b523947a948e786ad2129591880d9, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.194407 2061 consensus_queue.cc:260] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.194423 2060 consensus_queue.cc:260] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.196473 2039 heartbeater.cc:503] Master 127.21.27.190:39415 was elected leader, sending a full tablet report...
I20260809 22:45:47.197502 2060 ts_tablet_manager.cc:1428] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Time spent starting tablet: real 0.022s user 0.012s sys 0.008s
I20260809 22:45:47.201012 2061 ts_tablet_manager.cc:1428] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Time spent starting tablet: real 0.024s user 0.015s sys 0.009s
I20260809 22:45:47.201613 2065 raft_consensus.cc:348] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.202260 2065 raft_consensus.cc:374] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:45:47.202425 2065 raft_consensus.cc:729] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a5a586b00b4c87bcf0eeb92ac9ecfe, State: Initialized, Role: FOLLOWER
I20260809 22:45:47.203459 2065 consensus_queue.cc:260] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.206138 2065 ts_tablet_manager.cc:1428] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Time spent starting tablet: real 0.019s user 0.013s sys 0.005s
W20260809 22:45:47.264380 2040 tablet.cc:2369] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:45:47.330128 1720 tablet.cc:2369] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:45:47.376698 1884 tablet.cc:2369] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:45:47.451774 2067 raft_consensus.cc:482] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:45:47.452339 2067 raft_consensus.cc:504] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.454592 2067 leader_election.cc:288] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0d681f4802244e8bf7e524becbde861 (127.21.27.130:35019), 10a5a586b00b4c87bcf0eeb92ac9ecfe (127.21.27.129:42467)
I20260809 22:45:47.463518 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.455000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:35019 (local address 127.21.27.131:49773)
0809 22:45:47.455251 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.455269 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.455386 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.456324 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.456327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.456381 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.456728 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.456738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.458329 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.458333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.460676 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.460698 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.462505 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.462509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.462903 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.462931 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.463014 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.463141 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 22:45:47.463763 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.455500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:49773 (local address 127.21.27.130:35019)
0809 22:45:47.455810 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.455816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.455843 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:45:47.455909 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.455913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.455970 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.456083 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.456879 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.458134 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.460896 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.462372 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.462575 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.462654 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.463114 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.463328 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:45:47.465090 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.455693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:42467 (local address 127.21.27.131:52631)
0809 22:45:47.456852 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0809 22:45:47.456868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:45:47.457066 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:45:47.457421 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:45:47.457424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:45:47.457443 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:45:47.458084 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.458094 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.459587 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.459593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.461968 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:45:47.462017 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.464076 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.464085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:45:47.464361 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.464410 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:45:47.464545 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 22:45:47.464702 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"thread_start_us":793,"threads_started":1}
I20260809 22:45:47.465117 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.455954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:52631 (local address 127.21.27.129:42467)
0809 22:45:47.456222 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.456228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.457051 (+ 823us) server_negotiation.cc:407] Connection header received
0809 22:45:47.457183 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.457188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.457277 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.457410 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:45:47.458279 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.459397 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.462252 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.463810 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.464014 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.464103 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.464614 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.464777 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:45:47.465480 1833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abc25e732314858a0920cb704b40e99" candidate_uuid: "8e8b523947a948e786ad2129591880d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d681f4802244e8bf7e524becbde861" is_pre_election: true
I20260809 22:45:47.466424 1833 raft_consensus.cc:2454] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8b523947a948e786ad2129591880d9 in term 0.
I20260809 22:45:47.466501 1670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abc25e732314858a0920cb704b40e99" candidate_uuid: "8e8b523947a948e786ad2129591880d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" is_pre_election: true
I20260809 22:45:47.467439 1670 raft_consensus.cc:2454] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e8b523947a948e786ad2129591880d9 in term 0.
I20260809 22:45:47.467592 1909 leader_election.cc:302] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e8b523947a948e786ad2129591880d9, d0d681f4802244e8bf7e524becbde861; no voters:
I20260809 22:45:47.468176 2067 raft_consensus.cc:2781] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:45:47.468449 2067 raft_consensus.cc:482] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:45:47.468602 2067 raft_consensus.cc:3037] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.471813 2067 raft_consensus.cc:504] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.472514 2067 leader_election.cc:288] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [CANDIDATE]: Term 1 election: Requested vote from peers d0d681f4802244e8bf7e524becbde861 (127.21.27.130:35019), 10a5a586b00b4c87bcf0eeb92ac9ecfe (127.21.27.129:42467)
I20260809 22:45:47.473292 1833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abc25e732314858a0920cb704b40e99" candidate_uuid: "8e8b523947a948e786ad2129591880d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d681f4802244e8bf7e524becbde861"
I20260809 22:45:47.473407 1670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abc25e732314858a0920cb704b40e99" candidate_uuid: "8e8b523947a948e786ad2129591880d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe"
I20260809 22:45:47.473567 1833 raft_consensus.cc:3037] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.473767 1670 raft_consensus.cc:3037] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:45:47.476856 1833 raft_consensus.cc:2454] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8b523947a948e786ad2129591880d9 in term 1.
I20260809 22:45:47.477186 1670 raft_consensus.cc:2454] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e8b523947a948e786ad2129591880d9 in term 1.
I20260809 22:45:47.477499 1909 leader_election.cc:302] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e8b523947a948e786ad2129591880d9, d0d681f4802244e8bf7e524becbde861; no voters:
I20260809 22:45:47.478017 2067 raft_consensus.cc:2781] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:45:47.478883 2067 raft_consensus.cc:686] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 1 LEADER]: Becoming Leader. State: Replica: 8e8b523947a948e786ad2129591880d9, State: Running, Role: LEADER
I20260809 22:45:47.479573 2067 consensus_queue.cc:237] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } }
I20260809 22:45:47.485188 1478 catalog_manager.cc:5495] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e8b523947a948e786ad2129591880d9 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "8e8b523947a948e786ad2129591880d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e8b523947a948e786ad2129591880d9" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43167 } health_report { overall_health: HEALTHY } } }
I20260809 22:45:47.510550 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.503260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.21.27.131:43167)
0809 22:45:47.503486 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.503491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.503508 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:45:47.503596 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.503599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.503719 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.503866 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:45:47.504317 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.505253 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.507481 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.508083 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.508225 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.508976 (+ 751us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:45:47.509572 (+ 596us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:45:47.509691 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.509917 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.510015 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:45:47.521406 1670 raft_consensus.cc:1261] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 1 FOLLOWER]: Refusing update from remote peer 8e8b523947a948e786ad2129591880d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:47.521463 1833 raft_consensus.cc:1261] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 1 FOLLOWER]: Refusing update from remote peer 8e8b523947a948e786ad2129591880d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:45:47.522639 2073 consensus_queue.cc:1035] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a5a586b00b4c87bcf0eeb92ac9ecfe" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 42467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.523306 2067 consensus_queue.cc:1035] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0d681f4802244e8bf7e524becbde861" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 35019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:45:47.540051 2076 mvcc.cc:205] Tried to move back new op lower bound from 7316748482634944512 to 7316748482481471488. Current Snapshot: MvccSnapshot[applied={T|T < 7316748482634944512}]
I20260809 22:45:47.542407 2077 mvcc.cc:205] Tried to move back new op lower bound from 7316748482634944512 to 7316748482481471488. Current Snapshot: MvccSnapshot[applied={T|T < 7316748482634944512}]
I20260809 22:45:47.558199 1478 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:47.558754 1478 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:45:47.564229 1478 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bae2c6656db24d08b1d495e240c087ac: Sending DeleteTablet for 3 replicas of tablet 5abc25e732314858a0920cb704b40e99
I20260809 22:45:47.565819 1807 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abc25e732314858a0920cb704b40e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:47738
I20260809 22:45:47.565855 1646 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abc25e732314858a0920cb704b40e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:44786
I20260809 22:45:47.566505 1964 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abc25e732314858a0920cb704b40e99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:45:47 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:35814
I20260809 22:45:47.568728 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 1569
I20260809 22:45:47.578789 2085 tablet_replica.cc:330] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: stopping tablet replica
I20260809 22:45:47.579666 2085 raft_consensus.cc:2229] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:47.580509 2085 raft_consensus.cc:2258] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.582217 2086 tablet_replica.cc:330] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: stopping tablet replica
I20260809 22:45:47.582474 2087 tablet_replica.cc:330] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: stopping tablet replica
I20260809 22:45:47.582983 2087 raft_consensus.cc:2229] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:45:47.583787 2087 raft_consensus.cc:2258] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.584403 2085 ts_tablet_manager.cc:1905] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:47.584928 2086 raft_consensus.cc:2229] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:45:47.586118 2087 ts_tablet_manager.cc:1905] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:45:47.588039 2086 raft_consensus.cc:2258] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:45:47.590808 2086 ts_tablet_manager.cc:1905] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 22:45:47 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:45:47.594511 2085 ts_tablet_manager.cc:1918] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:47.594933 2085 log.cc:1196] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/5abc25e732314858a0920cb704b40e99
I20260809 22:45:47.595893 2085 ts_tablet_manager.cc:1939] T 5abc25e732314858a0920cb704b40e99 P d0d681f4802244e8bf7e524becbde861: Deleting consensus metadata
I20260809 22:45:47.598774 1461 catalog_manager.cc:4841] TS d0d681f4802244e8bf7e524becbde861 (127.21.27.130:35019): tablet 5abc25e732314858a0920cb704b40e99 (table test-table [id=8bfdf70ba0684468af94b3301f16432e]) successfully deleted
I20260809 22:45:47.603439 2087 ts_tablet_manager.cc:1918] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:47.603883 2087 log.cc:1196] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/5abc25e732314858a0920cb704b40e99
I20260809 22:45:47.604136 2086 ts_tablet_manager.cc:1918] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:45:47.604606 2086 log.cc:1196] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/5abc25e732314858a0920cb704b40e99
I20260809 22:45:47.604847 2087 ts_tablet_manager.cc:1939] T 5abc25e732314858a0920cb704b40e99 P 8e8b523947a948e786ad2129591880d9: Deleting consensus metadata
I20260809 22:45:47.605579 2086 ts_tablet_manager.cc:1939] T 5abc25e732314858a0920cb704b40e99 P 10a5a586b00b4c87bcf0eeb92ac9ecfe: Deleting consensus metadata
I20260809 22:45:47.606365 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.570819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.21.27.129:42467)
0809 22:45:47.571061 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.571070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.580254 (+ 9184us) server_negotiation.cc:407] Connection header received
0809 22:45:47.580800 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.580807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.580911 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.580993 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.583189 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.584401 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.586458 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.587124 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.587283 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.594488 (+ 7205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.594517 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.594555 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.594631 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.601068 (+ 6437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.602441 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.602451 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.602455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.602681 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.603380 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.603389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.603394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.604166 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.604633 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.605393 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.605729 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:45:47.607951 1459 catalog_manager.cc:4841] TS 8e8b523947a948e786ad2129591880d9 (127.21.27.131:43167): tablet 5abc25e732314858a0920cb704b40e99 (table test-table [id=8bfdf70ba0684468af94b3301f16432e]) successfully deleted
I20260809 22:45:47.608356 1461 catalog_manager.cc:4841] TS 10a5a586b00b4c87bcf0eeb92ac9ecfe (127.21.27.129:42467): tablet 5abc25e732314858a0920cb704b40e99 (table test-table [id=8bfdf70ba0684468af94b3301f16432e]) successfully deleted
I20260809 22:45:47.608456 1713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:47.707012 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 1569
I20260809 22:45:47.730171 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 1726
Aug 09 22:45:47 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:45:47.750077 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.731189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.21.27.130:35019)
0809 22:45:47.731417 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.731424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.731756 (+ 332us) server_negotiation.cc:407] Connection header received
0809 22:45:47.732017 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.732023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.732146 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.732255 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.734213 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.735440 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.737522 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.738031 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.738142 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.743558 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.743573 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.743589 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.743631 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.746751 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.747612 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.747620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.747622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.747834 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.748280 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.748285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.748287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.748733 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.749068 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.749406 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.749705 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 22:45:47.750824 1875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:47.844339 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 1726
I20260809 22:45:47.864635 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 1889
Aug 09 22:45:47 dist-test-slave-chsk krb5kdc[939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:45:47.883040 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:45:47.865491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.21.27.131:43167)
0809 22:45:47.865690 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:45:47.865697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:45:47.865726 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:45:47.865958 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:45:47.865964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:45:47.866064 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:45:47.866190 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:45:47.867674 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.868517 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.869872 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:45:47.870388 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:45:47.870566 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:45:47.876099 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:45:47.876111 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:45:47.876126 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:45:47.876181 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:45:47.879803 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.880759 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.880792 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.880797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.880968 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:45:47.881424 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:45:47.881427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:45:47.881429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:45:47.881881 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:45:47.882131 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:45:47.882537 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 22:45:47.882732 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:45:47.883905 2033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:48.008283 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 1889
I20260809 22:45:48.031248 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 1442
I20260809 22:45:48.032582 1511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:45:48.152356 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 1442
I20260809 22:45:48.181044 21614 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:45:48.527230 21614 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:45:48.527547 21614 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:45:48.873124 21614 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:45:48.873 UTC [975] LOG: received smart shutdown request
2026-08-09 22:45:48.878 UTC [975] LOG: background worker "logical replication launcher" (PID 986) exited with exit code 1
2026-08-09 22:45:48.878 UTC [980] LOG: shutting down
2026-08-09 22:45:49.043 UTC [975] LOG: database system is shut down
2026-08-09T22:45:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53433 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:45:49 dist-test-slave-chsk krb5kdc[2102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:45:49 dist-test-slave-chsk krb5kdc[2102](info): set up 2 sockets
Aug 09 22:45:49 dist-test-slave-chsk krb5kdc[2102](info): commencing operation
krb5kdc: starting...
W20260809 22:45:50.356300 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:45:50 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:45:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:45:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangeradmin_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangeradmin_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangerlookup_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangerlookup_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
I20260809 22:45:50.746073 21614 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvPTdCg/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/postgres -l logfile start
2026-08-09 22:45:53.284 UTC [2134] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 22:45:53.284 UTC [2134] LOG: listening on IPv4 address "127.21.27.148", port 36815
2026-08-09 22:45:53.287 UTC [2134] LOG: listening on Unix socket "/tmp/.s.PGSQL.36815"
2026-08-09 22:45:53.310 UTC [2137] LOG: database system was shut down at 2026-08-09 22:45:51 UTC
2026-08-09 22:45:53.314 UTC [2134] LOG: database system is ready to accept connections
I20260809 22:45:54.339697 21614 mini_postgres.cc:96] Postgres bound to 36815
2026-08-09 22:45:54.360 UTC [2146] FATAL: database "slave" does not exist
127.21.27.148:36815 - accepting connections
I20260809 22:45:54.361629 21614 mini_ranger.cc:161] Starting Ranger...
I20260809 22:45:54.391824 21614 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 22:45:54.873121 21614 mini_ranger.cc:87] Created ranger Postgres database
I20260809 22:45:54.873257 21614 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin
2026-08-09 22:45:55,178 [I] DB FLAVOR :POSTGRES
2026-08-09 22:45:55,179 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:45:55,179 [I] Checking connection..
2026-08-09 22:45:55,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 22:45:55,450 [I] Checking connection passed.
2026-08-09 22:45:55,450 [I] --------- Verifying version history table ---------
2026-08-09 22:45:55,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:55,716 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:45:55,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:55,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 22:45:55,989 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 22:45:55,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 22:45:56.257 UTC [2221] WARNING: there is no transaction in progress
2026-08-09 22:45:56,265 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 22:45:56,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 22:45:56,514 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 22:45:56,515 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:45:56,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 22:45:56,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 22:45:57,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-chsk.c.gcp-upstream.internal','N') ;"
2026-08-09 22:45:57,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:45:57,582 [I] Table x_portal_user does not exist in database ranger
2026-08-09 22:45:57,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:45:57,840 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 22:45:57,841 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:45:58,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:45:58,344 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 22:45:58,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 22:45:59.317 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.333 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.349 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.364 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.384 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.404 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.466 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.473 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.488 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.499 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.509 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.523 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.534 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.540 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.547 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:45:59.553 UTC [2349] WARNING: there is no transaction in progress
2026-08-09 22:46:00,202 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 22:46:00,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 22:46:00,466 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 22:46:00,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 22:46:00,739 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 22:46:00,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 22:46:01,010 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 22:46:01,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 22:46:01,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 22:46:01,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-chsk.c.gcp-upstream.internal';"
2026-08-09 22:46:01,768 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 22:46:01,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 22:46:02,040 [I] DB_PATCHES have already been applied
I20260809 22:46:02.047101 21614 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvPTdCg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 22:46:02.047304 21614 mini_ranger.cc:193] Using host: 127.21.27.148
I20260809 22:46:02.066879 21614 mini_ranger.cc:227] Ranger admin URL: http://127.21.27.148:47687
Aug 09, 2026 10:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 10:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 10:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 10:46:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 10:46:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 22:46:03.280279 21614 mini_ranger.cc:160] Time spent starting Ranger: real 8.919s user 0.000s sys 0.058s
Aug 09, 2026 10:46:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 10:46:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 10:46:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 10:46:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 10:46:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 22:46:25.465814 21614 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.27.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.27.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangerkms_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.27.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/rangerkms_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.27.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.27.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.27.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/HTTP_127.21.27.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 22:46:25 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315585, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 22:46:25.668411 21614 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 22:46:25.693326 21614 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 22:46:26.522713 21614 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 22:46:26.522812 21614 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms
I20260809 22:46:26.522872 21614 mini_ranger_kms.cc:227] Using postgres at 127.21.27.148:36815
2026-08-09 22:46:26,890 [I] DB FLAVOR :POSTGRES
2026-08-09 22:46:26,890 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 22:46:26,890 [I] Checking connection
2026-08-09 22:46:26,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 22:46:27,145 [I] connection success
2026-08-09 22:46:27,145 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 22:46:27,145 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 22:46:27,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 22:46:27,397 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 22:46:27,398 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 22:46:27,398 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 22:46:27,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.27.148:36815/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 22:46:27,674 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 22:46:28.179916 21614 mini_ranger_kms.cc:310] Created kms service
I20260809 22:46:28.380784 21614 mini_ranger_kms.cc:326] Created kudu user
I20260809 22:46:28.426743 21614 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 22:46:28 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315585, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:28.829609 21614 mini_ranger_kms.cc:384] Added ranger policy
I20260809 22:46:28.829964 21614 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvPTdCg/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvPTdCg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 22:46:28.830039 21614 mini_ranger_kms.cc:240] Using host: 127.21.27.148
I20260809 22:46:28.852381 21614 mini_ranger_kms.cc:276] Ranger KMS PID: 2552
I20260809 22:46:28.852618 21614 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.27.148:41081
Aug 09, 2026 10:46:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 10:46:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 10:46:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 10:46:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 10:46:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 10:46:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 22:46:31 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:31.302017 21614 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.634s user 0.095s sys 0.303s
I20260809 22:46:31.302280 21614 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 22:46:31.302479 21614 mini_ranger_kms.cc:401] 127.21.27.148:41081/kms/v1/keys
Aug 09 22:46:31 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
Aug 09 22:46:31 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315591, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.27.148@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
Aug 09 22:46:34 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:46:34.270826 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45017
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45017
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:34.612295 2605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:34.612615 2605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:34.612725 2605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:34.620026 2605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:34.620146 2605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:34.620191 2605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:34.620249 2605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:34.620296 2605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:34.630533 2605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.27.148:41081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45017
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:34.631853 2605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:34.633162 2605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:34.645366 2611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:34.645447 2610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:34.645421 2613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:34.646283 2605 server_base.cc:1034] running on GCE node
I20260809 22:46:34.647265 2605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:34.648818 2605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:34.650066 2605 hybrid_clock.cc:648] HybridClock initialized: now 1786315594650020 us; error 63 us; skew 500 ppm
I20260809 22:46:34.650446 2605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:34 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:34.653836 2605 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:34.655419 2605 webserver.cc:460] Webserver started at http://127.21.27.190:44151/ using document root <none> and password file <none>
I20260809 22:46:34.656088 2605 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:34.656240 2605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:34.656569 2605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:46:34 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:34.718392 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0fb608ca279440f5a33f46168bf57013"
format_stamp: "Formatted at 2026-08-09 22:46:34 on dist-test-slave-chsk"
server_key: "2c7f15370154c48df9c49e3add2b146a"
server_key_iv: "d92fff438d0ed2475de448783bc670ec"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:34.719447 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0fb608ca279440f5a33f46168bf57013"
format_stamp: "Formatted at 2026-08-09 22:46:34 on dist-test-slave-chsk"
server_key: "2c7f15370154c48df9c49e3add2b146a"
server_key_iv: "d92fff438d0ed2475de448783bc670ec"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:34.725162 2605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:46:34.742062 2620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:34.743671 2605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:34.743887 2605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0fb608ca279440f5a33f46168bf57013"
format_stamp: "Formatted at 2026-08-09 22:46:34 on dist-test-slave-chsk"
server_key: "2c7f15370154c48df9c49e3add2b146a"
server_key_iv: "d92fff438d0ed2475de448783bc670ec"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:34.744163 2605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:34.760077 2605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:34.763759 2605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:34.764101 2605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:34.784317 2605 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45017
I20260809 22:46:34.784340 2679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45017 every 8 connection(s)
I20260809 22:46:34.785737 2605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:34.789619 2680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:34.789741 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 2605
I20260809 22:46:34.789958 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:46:34.790632 21614 external_mini_cluster.cc:1428] Setting key 06553f1d2b7eeea7d3eeb410f7013e40
I20260809 22:46:34.804018 2680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Bootstrap starting.
Aug 09 22:46:34 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:34.807708 2680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:34.809180 2680 log.cc:824] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:34.812774 2680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: No bootstrap required, opened a new log
I20260809 22:46:34.815369 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:34.792679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.21.27.190:45017)
0809 22:46:34.793545 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:34.793562 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:34.793604 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:46:34.795734 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:34.795755 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:34.796158 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:34.796481 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:34.798274 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.799641 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.801524 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:34.802216 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:34.802344 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:34.807062 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:34.807070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:34.807084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:34.807126 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:34.810860 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.811581 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.811594 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.811601 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.811803 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:34.812164 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:34.812168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:34.812171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:34.812910 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:34.813264 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:34.813615 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 22:46:34.814003 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":408,"threads_started":1}
I20260809 22:46:34.821800 2680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:34.822294 2680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:34.822415 2680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb608ca279440f5a33f46168bf57013, State: Initialized, Role: FOLLOWER
I20260809 22:46:34.823022 2680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:34.823339 2680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:34.823486 2680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:34.823668 2680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:34.825906 2680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:34.826491 2680 leader_election.cc:302] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fb608ca279440f5a33f46168bf57013; no voters:
I20260809 22:46:34.827286 2680 leader_election.cc:288] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:34.827438 2685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:34.828244 2685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 LEADER]: Becoming Leader. State: Replica: 0fb608ca279440f5a33f46168bf57013, State: Running, Role: LEADER
I20260809 22:46:34.828905 2685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:34.829682 2680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:34.831911 2687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fb608ca279440f5a33f46168bf57013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } } }
I20260809 22:46:34.832454 2687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.833248 2686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fb608ca279440f5a33f46168bf57013. Latest consensus state: current_term: 1 leader_uuid: "0fb608ca279440f5a33f46168bf57013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } } }
I20260809 22:46:34.833701 2686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:34.833930 2693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:34.839495 2693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:34.845150 2680 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:46:34.860026 2693 catalog_manager.cc:1347] Generated new cluster ID: 613cef2fd04e4fcf91515fb47ba7e20a
I20260809 22:46:34.860244 2693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:34.887832 2693 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:34.889266 2693 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:34.901410 2693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Generated new TSK 0
I20260809 22:46:34.902585 2693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:46:35.703477 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:35 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 22:46:36.157487 2727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:36.157936 2727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:36.158159 2727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:36.170238 2727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:36.170485 2727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:36.170734 2727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:46:36.187338 2727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:36.189079 2727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:36.190766 2727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:36.208751 2740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:36.216557 2727 server_base.cc:1034] running on GCE node
W20260809 22:46:36.217007 2741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:36.217669 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:36.218606 2727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:36.223407 2727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:36.224714 2727 hybrid_clock.cc:648] HybridClock initialized: now 1786315596224654 us; error 65 us; skew 500 ppm
I20260809 22:46:36.225215 2727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:36 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:36.229678 2727 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:46:36.234107 2727 webserver.cc:460] Webserver started at http://127.21.27.129:39273/ using document root <none> and password file <none>
I20260809 22:46:36.234956 2727 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:36.235122 2727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:36.235482 2727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:46:36 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:36.255453 2727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "dfe33b84fc2647be9f675363dddaa5ab"
format_stamp: "Formatted at 2026-08-09 22:46:36 on dist-test-slave-chsk"
server_key: "cf740432f565b313b2e4b8c8db1fd653"
server_key_iv: "3a5fef6481504f06c706c239700f9f01"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:36.256649 2727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "dfe33b84fc2647be9f675363dddaa5ab"
format_stamp: "Formatted at 2026-08-09 22:46:36 on dist-test-slave-chsk"
server_key: "cf740432f565b313b2e4b8c8db1fd653"
server_key_iv: "3a5fef6481504f06c706c239700f9f01"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:36.265197 2727 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260809 22:46:36.284895 2750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:36.286950 2727 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260809 22:46:36.287283 2727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "dfe33b84fc2647be9f675363dddaa5ab"
format_stamp: "Formatted at 2026-08-09 22:46:36 on dist-test-slave-chsk"
server_key: "cf740432f565b313b2e4b8c8db1fd653"
server_key_iv: "3a5fef6481504f06c706c239700f9f01"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:36.287714 2727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:36.345430 2727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:36.351039 2727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:36.351490 2727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:36.352833 2727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:36.355927 2727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:36.356113 2727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 22:46:36.356323 2727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:36.356467 2727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:36.427420 2727 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:35777
I20260809 22:46:36.427456 2871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:35777 every 8 connection(s)
I20260809 22:46:36.428954 2727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:36.431001 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 2727
I20260809 22:46:36.431284 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:36.432246 21614 external_mini_cluster.cc:1428] Setting key e55e2e18df4f993998ce92e2f135fc79
Aug 09 22:46:36 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:36.483075 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.434586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:58943 (local address 127.21.27.190:45017)
0809 22:46:36.455680 (+ 21094us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:36.455690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:36.457027 (+ 1337us) server_negotiation.cc:407] Connection header received
0809 22:46:36.461943 (+ 4916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:36.461949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:36.462097 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:36.462219 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:36.464643 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.466022 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.468713 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.469362 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.469466 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.474748 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:36.474757 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:36.474765 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:36.474827 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:36.478660 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.479906 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.479912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.479915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.480098 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:36.480696 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:36.480702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:36.480705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:36.481082 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:36.481226 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:36.482287 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0809 22:46:36.482543 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20884,"thread_start_us":20739,"threads_started":1}
I20260809 22:46:36.485906 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:36.433350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.129:58943)
0809 22:46:36.456384 (+ 23034us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:36.456441 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:36.461374 (+ 4933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:36.462750 (+ 1376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:36.462764 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:36.463520 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:36.464390 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.464419 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.466252 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.466258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.468498 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:36.468541 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:36.469869 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:36.469877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:36.470171 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:36.471099 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:36.471156 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:36.474430 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:36.479035 (+ 4605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.479048 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.479069 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.479749 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.480268 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:36.480275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:36.480279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:36.480554 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:36.481327 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:36.481335 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:36.481797 (+ 462us) client_negotiation.cc:769] Sending connection context
0809 22:46:36.483702 (+ 1905us) client_negotiation.cc:240] Negotiation successful
0809 22:46:36.484374 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22708,"thread_start_us":22419,"threads_started":1}
I20260809 22:46:36.488126 2872 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:36.488408 2872 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:36.489432 2872 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:36.492017 2639 ts_manager.cc:194] Registered new tserver with Master: dfe33b84fc2647be9f675363dddaa5ab (127.21.27.129:35777)
I20260809 22:46:36.494031 2639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:58943
Aug 09 22:46:36 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
W20260809 22:46:36.553031 21614 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.27.130: real 0.113s user 0.009s sys 0.013s
I20260809 22:46:36.554010 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:36 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
W20260809 22:46:36.987483 2880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:36.988009 2880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:36.988214 2880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:37.002206 2880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:37.019344 2880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:37.019660 2880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:46:37.040459 2880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:37.042464 2880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:37.044307 2880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:37.072458 2889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:37.079862 2888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:37.081718 2880 server_base.cc:1034] running on GCE node
W20260809 22:46:37.084295 2891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:37.085260 2880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:37.087513 2880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:37.088830 2880 hybrid_clock.cc:648] HybridClock initialized: now 1786315597088717 us; error 99 us; skew 500 ppm
I20260809 22:46:37.089289 2880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:37.094444 2880 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:46:37.103556 2880 webserver.cc:460] Webserver started at http://127.21.27.130:42961/ using document root <none> and password file <none>
I20260809 22:46:37.104712 2880 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:37.104915 2880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:37.105362 2880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:37.126749 2880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "3340ce2bf20041948d4f407dd70b5cf2"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "ede20462da17f6e1bb05e848f36d5fa6"
server_key_iv: "1ca1247b26353ff7674aaae1fff35ef9"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.127847 2880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "3340ce2bf20041948d4f407dd70b5cf2"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "ede20462da17f6e1bb05e848f36d5fa6"
server_key_iv: "1ca1247b26353ff7674aaae1fff35ef9"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.133910 2880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 22:46:37.157907 2899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:37.159848 2880 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.005s
I20260809 22:46:37.160084 2880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "3340ce2bf20041948d4f407dd70b5cf2"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "ede20462da17f6e1bb05e848f36d5fa6"
server_key_iv: "1ca1247b26353ff7674aaae1fff35ef9"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.160368 2880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:37.198206 2880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:37.203549 2880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:37.203876 2880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:37.205102 2880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:37.206935 2880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:37.207037 2880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:37.207207 2880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:37.207283 2880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 22:46:37.258179 2880 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:34273
I20260809 22:46:37.258230 3029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:34273 every 8 connection(s)
I20260809 22:46:37.259742 2880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:37.269445 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 2880
I20260809 22:46:37.269793 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:37.270707 21614 external_mini_cluster.cc:1428] Setting key c7c82e48f03ddccb912fc262d947758c
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:37.290380 3032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.262592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:36403 (local address 127.21.27.190:45017)
0809 22:46:37.266277 (+ 3685us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:37.266284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:37.268688 (+ 2404us) server_negotiation.cc:407] Connection header received
0809 22:46:37.271630 (+ 2942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:37.271636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:37.271737 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:37.271847 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:37.274003 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.275109 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.277541 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.278183 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.278294 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.282888 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:37.282894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:37.282899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:37.282939 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:37.286550 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.287532 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.287538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.287542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.287689 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.288219 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.288224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.288227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.288643 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:37.288771 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:37.289709 (+ 938us) server_negotiation.cc:299] Negotiation successful
0809 22:46:37.289940 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3541,"thread_start_us":3411,"threads_started":1}
I20260809 22:46:37.292800 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.263469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.130:36403)
0809 22:46:37.268424 (+ 4955us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.268479 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:37.271215 (+ 2736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:37.272284 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:37.272296 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:37.272976 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:37.273800 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.273826 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.275337 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.275342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.277364 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.277389 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.278626 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.278634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.278932 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.279771 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:37.279801 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:37.282635 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:37.286724 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:37.286733 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:37.286750 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:37.287375 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:37.287838 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:37.287843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:37.287846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:37.288072 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:37.288870 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:37.288877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:37.289280 (+ 403us) client_negotiation.cc:769] Sending connection context
0809 22:46:37.290892 (+ 1612us) client_negotiation.cc:240] Negotiation successful
0809 22:46:37.291447 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4665,"thread_start_us":4208,"threads_started":1}
I20260809 22:46:37.294706 3030 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:37.295020 3030 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:37.295913 3030 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:37.297664 2639 ts_manager.cc:194] Registered new tserver with Master: 3340ce2bf20041948d4f407dd70b5cf2 (127.21.27.130:34273)
I20260809 22:46:37.298946 2639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:36403
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:46:37.340353 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:37.497854 2872 heartbeater.cc:503] Master 127.21.27.190:45017 was elected leader, sending a full tablet report...
W20260809 22:46:37.726750 3037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:37.727030 3037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:37.727124 3037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:37.738226 3037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:37.738382 3037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:37.738605 3037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:46:37.749150 3037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:37.750344 3037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:37.751622 3037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:37.764487 3043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:37.764632 3045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:37.764997 3042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:37.765530 3037 server_base.cc:1034] running on GCE node
I20260809 22:46:37.766258 3037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:37.767333 3037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:37.768553 3037 hybrid_clock.cc:648] HybridClock initialized: now 1786315597768505 us; error 34 us; skew 500 ppm
I20260809 22:46:37.768874 3037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:37.772130 3037 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:46:37.773515 3037 webserver.cc:460] Webserver started at http://127.21.27.131:44043/ using document root <none> and password file <none>
I20260809 22:46:37.774159 3037 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:37.774292 3037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:37.774605 3037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:37.791051 3037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "edc16942d89f4788be52919e1eff0231"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "b9b5d11ebcda56141df69a6e7c9ef9c0"
server_key_iv: "15b66679f59cfe3ec0a610a0cd075c38"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.792352 3037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "edc16942d89f4788be52919e1eff0231"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "b9b5d11ebcda56141df69a6e7c9ef9c0"
server_key_iv: "15b66679f59cfe3ec0a610a0cd075c38"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.799571 3037 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 22:46:37.816818 3053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:37.818264 3037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:37.818427 3037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "edc16942d89f4788be52919e1eff0231"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "b9b5d11ebcda56141df69a6e7c9ef9c0"
server_key_iv: "15b66679f59cfe3ec0a610a0cd075c38"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:37.818620 3037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:37.849736 3037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:37.853487 3037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:37.853744 3037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:37.854604 3037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:37.856102 3037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:37.856176 3037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:37.856258 3037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:37.856307 3037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:37.890587 3037 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:44707
I20260809 22:46:37.890636 3173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:44707 every 8 connection(s)
I20260809 22:46:37.892328 3037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:37.901970 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3037
I20260809 22:46:37.902191 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:37.902658 21614 external_mini_cluster.cc:1428] Setting key 939ffb3496f07c3e37dcb04456b4d3ea
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315597, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:37.912209 3176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.895105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:48685 (local address 127.21.27.190:45017)
0809 22:46:37.895678 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:37.895682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:37.896645 (+ 963us) server_negotiation.cc:407] Connection header received
0809 22:46:37.898573 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:37.898579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:37.898665 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:37.898781 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:37.900247 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.901002 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.902751 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.903272 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.903408 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.906958 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:37.906963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:37.906966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:37.906999 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:37.909457 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.910133 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.910137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.910139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.910266 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.910687 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.910691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.910692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.910954 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:37.911061 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:37.911764 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 22:46:37.911934 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":246,"threads_started":1}
I20260809 22:46:37.913172 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.895760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.131:48685)
0809 22:46:37.896488 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:37.896523 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:37.898287 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:37.899018 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:37.899025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:37.899524 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:37.900072 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.900090 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.901192 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.901196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.902601 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:37.902622 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.903509 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.903517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:37.903789 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.904490 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:37.904514 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:37.906749 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:37.909590 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:37.909597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:37.909608 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:37.910030 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:37.910357 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:37.910360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:37.910362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:37.910555 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:37.911066 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:37.911071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:37.911425 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 22:46:37.911688 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 22:46:37.912052 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":312,"threads_started":1}
I20260809 22:46:37.914614 3174 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:37.914951 3174 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:37.915701 3174 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:37.917389 2639 ts_manager.cc:194] Registered new tserver with Master: edc16942d89f4788be52919e1eff0231 (127.21.27.131:44707)
I20260809 22:46:37.918231 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:37.918351 2639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:48685
I20260809 22:46:37.920671 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 2727
Aug 09 22:46:37 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:46:37.938313 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:37.921829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.21.27.129:35777)
0809 22:46:37.922604 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:37.922610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:37.922633 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:37.922692 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:37.922698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:37.922872 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:37.922993 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:37.924577 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.925553 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.927297 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:37.927839 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:37.927943 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:37.932510 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:37.932524 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:37.932538 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:37.932575 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:37.935434 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.936221 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.936228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.936230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.936391 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:37.936745 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:37.936748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:37.936750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:37.937173 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:37.937438 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:37.937705 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 22:46:37.937915 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":460,"threads_started":1}
I20260809 22:46:37.939704 2866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:38.023835 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 2727
I20260809 22:46:38.043999 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 2880
Aug 09 22:46:38 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:46:38.060410 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:38.044868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.21.27.130:34273)
0809 22:46:38.045409 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:38.045416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:38.045440 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:38.045500 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:38.045506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:38.045725 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:38.045893 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:38.047227 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:38.048357 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:38.049735 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:38.050276 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:38.050415 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:38.054719 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:38.054729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:38.054742 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:38.054773 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:38.057531 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:38.058197 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:38.058210 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:38.058212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:38.058343 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:38.058712 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:38.058717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:38.058725 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:38.059060 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:38.059411 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:38.059781 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 22:46:38.060056 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":273,"threads_started":1}
I20260809 22:46:38.061687 3024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:38.147776 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 2880
I20260809 22:46:38.171119 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3037
Aug 09 22:46:38 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:46:38.188814 3184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:38.172254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.21.27.131:44707)
0809 22:46:38.172939 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:38.172947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:38.172977 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:46:38.173068 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:38.173074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:38.173323 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:38.173456 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:38.174891 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:38.176147 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:38.177435 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:38.177987 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:38.178097 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:38.182918 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:38.182929 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:38.182944 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:38.182981 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:38.186229 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:38.186910 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:38.186916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:38.186918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:38.187050 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:38.187473 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:38.187477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:38.187479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:38.187824 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:38.188064 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:38.188343 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 22:46:38.188536 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":369,"threads_started":1}
I20260809 22:46:38.190145 3169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:38.274559 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3037
I20260809 22:46:38.293547 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 2605
I20260809 22:46:38.294836 2675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:38.405606 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 2605
I20260809 22:46:38.430883 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45017
--webserver_interface=127.21.27.190
--webserver_port=44151
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45017
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:38.762419 3188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:38.762753 3188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:38.762917 3188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:38.770215 3188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:38.770364 3188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:38.770491 3188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:38.770592 3188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:38.770677 3188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:38.780609 3188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45017
--ranger_config_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=44151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:38.781868 3188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:38.783164 3188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:38.794907 3194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:38.794864 3195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:38.794921 3197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:38.795662 3188 server_base.cc:1034] running on GCE node
I20260809 22:46:38.796401 3188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:38.797791 3188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:38.799016 3188 hybrid_clock.cc:648] HybridClock initialized: now 1786315598798982 us; error 48 us; skew 500 ppm
I20260809 22:46:38.799393 3188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:38 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:38.802352 3188 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:38.803833 3188 webserver.cc:460] Webserver started at http://127.21.27.190:44151/ using document root <none> and password file <none>
I20260809 22:46:38.804482 3188 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:38.804638 3188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:46:38 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315598, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:38.888110 3188 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:46:38.891189 3205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:38.892899 3188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:38.893118 3188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0fb608ca279440f5a33f46168bf57013"
format_stamp: "Formatted at 2026-08-09 22:46:34 on dist-test-slave-chsk"
server_key: "2c7f15370154c48df9c49e3add2b146a"
server_key_iv: "d92fff438d0ed2475de448783bc670ec"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:38.894515 3188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:38.913292 3188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:38.917197 3188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:38.917497 3188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:38.936923 3188 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45017
I20260809 22:46:38.936949 3267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45017 every 8 connection(s)
I20260809 22:46:38.938565 3188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:38.939836 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3188
I20260809 22:46:38.941017 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:35777
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:45017
--webserver_port=39273
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:38.948993 3268 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:38.953658 3268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Bootstrap starting.
I20260809 22:46:38.973446 3268 log.cc:824] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:38.983086 3268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:38.983768 3268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Bootstrap complete.
I20260809 22:46:38.993597 3268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:38.994553 3268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb608ca279440f5a33f46168bf57013, State: Initialized, Role: FOLLOWER
I20260809 22:46:38.995285 3268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:38.995545 3268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:38.995683 3268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:38.995864 3268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:46:39.000279 3268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:39.000834 3268 leader_election.cc:302] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fb608ca279440f5a33f46168bf57013; no voters:
I20260809 22:46:39.001519 3268 leader_election.cc:288] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:46:39.001681 3273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:46:39.004063 3268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:39.008706 3273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [term 2 LEADER]: Becoming Leader. State: Replica: 0fb608ca279440f5a33f46168bf57013, State: Running, Role: LEADER
I20260809 22:46:39.009475 3273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } }
I20260809 22:46:39.012697 3274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0fb608ca279440f5a33f46168bf57013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } } }
I20260809 22:46:39.013150 3274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:39.013049 3275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fb608ca279440f5a33f46168bf57013. Latest consensus state: current_term: 2 leader_uuid: "0fb608ca279440f5a33f46168bf57013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fb608ca279440f5a33f46168bf57013" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45017 } } }
I20260809 22:46:39.013410 3275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:39.014357 3291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:39.014995 3268 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 22:46:39.021227 3291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:39.023169 3291 catalog_manager.cc:1259] Loaded cluster ID: 613cef2fd04e4fcf91515fb47ba7e20a
I20260809 22:46:39.023295 3291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:39.034502 3291 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:39.037945 3291 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Loaded TSK: 0
I20260809 22:46:39.039788 3291 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:46:39.412508 3270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:39.412920 3270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:39.413110 3270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:39.425081 3270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:39.425220 3270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:39.425413 3270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:46:39.441197 3270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:35777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=39273
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:39.442862 3270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:39.444582 3270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:39.464566 3314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:39.464723 3313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:39.485204 3316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:39.487224 3270 server_base.cc:1034] running on GCE node
I20260809 22:46:39.488284 3270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:39.489648 3270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:39.490914 3270 hybrid_clock.cc:648] HybridClock initialized: now 1786315599490854 us; error 51 us; skew 500 ppm
I20260809 22:46:39.491397 3270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:39 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:39.497565 3270 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:46:39.500964 3270 webserver.cc:460] Webserver started at http://127.21.27.129:39273/ using document root <none> and password file <none>
I20260809 22:46:39.501967 3270 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:39.502168 3270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:46:39 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:39.527063 3270 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:46:39.532433 3324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:39.533942 3270 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260809 22:46:39.534178 3270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "dfe33b84fc2647be9f675363dddaa5ab"
format_stamp: "Formatted at 2026-08-09 22:46:36 on dist-test-slave-chsk"
server_key: "cf740432f565b313b2e4b8c8db1fd653"
server_key_iv: "3a5fef6481504f06c706c239700f9f01"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:39.535247 3270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:39.566517 3270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:39.572028 3270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:39.572345 3270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:39.574793 3270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:39.576731 3270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:39.576886 3270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:39.577030 3270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:39.577129 3270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:39.658481 3270 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:35777
I20260809 22:46:39.658591 3448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:35777 every 8 connection(s)
I20260809 22:46:39.660076 3270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:39.670439 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3270
I20260809 22:46:39.671352 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:34273
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:45017
--webserver_port=42961
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:39 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:39.693936 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.663623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.129:50921)
0809 22:46:39.664455 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:39.664522 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:39.666520 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:39.668066 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:39.668077 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:39.668643 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:39.669167 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.669185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.671532 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.671539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.673785 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:39.673804 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.675730 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.675739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:39.676060 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.677955 (+ 1895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:39.677991 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:39.681069 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:39.688429 (+ 7360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:39.688440 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:39.688457 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:39.689108 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:39.689705 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:39.689712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:39.689719 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:39.689998 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:39.691116 (+ 1118us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:39.691126 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:39.691571 (+ 445us) client_negotiation.cc:769] Sending connection context
0809 22:46:39.691894 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 22:46:39.692496 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":356,"threads_started":1}
I20260809 22:46:39.694968 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:39.663620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50921 (local address 127.21.27.190:45017)
0809 22:46:39.665311 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:39.665328 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:39.665381 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:46:39.667114 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:39.667126 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:39.667554 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:39.667908 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:39.669385 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.671304 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.673972 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:39.675088 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:39.675330 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:39.681417 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:39.681430 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:39.681450 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:39.681506 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:39.688206 (+ 6700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:39.689296 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:39.689308 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:39.689318 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:39.689537 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:39.690157 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:39.690164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:39.690170 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:39.690928 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:39.691124 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:39.693278 (+ 2154us) server_negotiation.cc:299] Negotiation successful
0809 22:46:39.693802 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1430,"thread_start_us":1207,"threads_started":1}
I20260809 22:46:39.698117 3449 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:39.698448 3449 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:39.699401 3449 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
W20260809 22:46:39.702829 3449 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:45017 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:46:39.703065 3449 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:39.703377 3449 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:39.704541 3449 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:39.704811 3449 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
W20260809 22:46:39.705871 3449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:46:40.146528 3454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:40.146929 3454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:40.147188 3454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:40.156052 3454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:40.156158 3454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:40.156385 3454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:46:40.166532 3454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:34273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=42961
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:40.167886 3454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:40.169154 3454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:40.184669 3467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:40.185737 3468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:40.187234 3454 server_base.cc:1034] running on GCE node
W20260809 22:46:40.187875 3471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:40.188926 3454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:40.190282 3454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:40.191569 3454 hybrid_clock.cc:648] HybridClock initialized: now 1786315600191508 us; error 45 us; skew 500 ppm
I20260809 22:46:40.191972 3454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:40.196102 3454 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:46:40.200605 3454 webserver.cc:460] Webserver started at http://127.21.27.130:42961/ using document root <none> and password file <none>
I20260809 22:46:40.201501 3454 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:40.201682 3454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:40.226344 3454 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:46:40.229445 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:40.230993 3454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:46:40.231212 3454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "3340ce2bf20041948d4f407dd70b5cf2"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "ede20462da17f6e1bb05e848f36d5fa6"
server_key_iv: "1ca1247b26353ff7674aaae1fff35ef9"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:40.232297 3454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:40.245121 3454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:40.249636 3454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:40.249943 3454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:40.251549 3454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:40.253369 3454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:40.253489 3454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:40.253598 3454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:40.253688 3454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:40.458635 3454 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:34273
I20260809 22:46:40.458729 3616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:34273 every 8 connection(s)
I20260809 22:46:40.460399 3454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:40.470373 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3454
I20260809 22:46:40.471313 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.27.148:41081/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:44707
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:45017
--webserver_port=44043
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:40.496903 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.464526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:41587 (local address 127.21.27.190:45017)
0809 22:46:40.471222 (+ 6696us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:40.471229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:40.471252 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:40.471319 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:40.471324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:40.471413 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:40.471530 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:40.474640 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.475900 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.478578 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.479371 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.479520 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:40.488247 (+ 8727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:40.488259 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:40.488266 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:40.488324 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:40.492387 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.493483 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.493492 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.493498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.493667 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:40.494294 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:40.494302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:40.494308 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:40.494730 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:40.494875 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:40.496073 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0809 22:46:40.496369 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6533,"thread_start_us":10758,"threads_started":1}
I20260809 22:46:40.500082 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:40.463888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.130:41587)
0809 22:46:40.467318 (+ 3430us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:40.467374 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:40.470469 (+ 3095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:40.472815 (+ 2346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:40.472830 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:40.473577 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:40.474391 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:40.474420 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.476163 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.476171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:40.478378 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:40.478405 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:40.479637 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:40.479646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:40.479926 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:40.484815 (+ 4889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:40.484856 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:40.487910 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:40.492596 (+ 4686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:40.492609 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:40.492632 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:40.493329 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:40.493838 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:40.493845 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:40.493851 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:40.494152 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:40.495000 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:40.495012 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:40.495560 (+ 548us) client_negotiation.cc:769] Sending connection context
0809 22:46:40.497558 (+ 1998us) client_negotiation.cc:240] Negotiation successful
0809 22:46:40.498244 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3101,"thread_start_us":2862,"threads_started":1}
I20260809 22:46:40.502581 3618 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:40.502931 3618 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:40.503971 3618 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:40.506147 3223 ts_manager.cc:194] Registered new tserver with Master: 3340ce2bf20041948d4f407dd70b5cf2 (127.21.27.130:34273)
I20260809 22:46:40.508296 3223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:41587
I20260809 22:46:40.706094 3449 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:40.706697 3449 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:40.708422 3223 ts_manager.cc:194] Registered new tserver with Master: dfe33b84fc2647be9f675363dddaa5ab (127.21.27.129:35777)
I20260809 22:46:40.709834 3223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:50921
W20260809 22:46:40.879117 3622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:40.879500 3622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:40.879643 3622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
W20260809 22:46:40.891942 3622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:40.892091 3622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:40.892288 3622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:46:40.909015 3622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.27.148:41081/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:44707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=44043
--tserver_master_addrs=127.21.27.190:45017
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:40.910869 3622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:40.912699 3622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:40.953394 3629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:40.953617 3628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:40.958657 3631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:40.959697 3622 server_base.cc:1034] running on GCE node
I20260809 22:46:40.960556 3622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:40.961918 3622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:40.963265 3622 hybrid_clock.cc:648] HybridClock initialized: now 1786315600963174 us; error 63 us; skew 500 ppm
I20260809 22:46:40.963719 3622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:40.968130 3622 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:46:40.972673 3622 webserver.cc:460] Webserver started at http://127.21.27.131:44043/ using document root <none> and password file <none>
I20260809 22:46:40.973884 3622 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:40.974053 3622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 22:46:40 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
I20260809 22:46:41.004736 3622 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 22:46:41.011777 3640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:41.013548 3622 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.005s
I20260809 22:46:41.013746 3622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "edc16942d89f4788be52919e1eff0231"
format_stamp: "Formatted at 2026-08-09 22:46:37 on dist-test-slave-chsk"
server_key: "b9b5d11ebcda56141df69a6e7c9ef9c0"
server_key_iv: "15b66679f59cfe3ec0a610a0cd075c38"
server_key_version: "kuduclusterkey@0"
I20260809 22:46:41.014954 3622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:41.048013 3622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:41.053640 3622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:41.053982 3622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:41.055440 3622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:41.057315 3622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:41.057428 3622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:41.057533 3622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:41.057618 3622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:41.100786 3622 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:44707
I20260809 22:46:41.100837 3764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:44707 every 8 connection(s)
I20260809 22:46:41.102874 3622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:41.107793 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3622
Aug 09 22:46:41 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:41.141710 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.106822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45017 (local address 127.21.27.131:48583)
0809 22:46:41.115759 (+ 8937us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.115826 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.118872 (+ 3046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.120049 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.120061 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.120616 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:41.121429 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.121458 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.123212 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.123217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.125054 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.125068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.126403 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.126410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.126681 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.127384 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:41.127420 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:41.130193 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:41.135105 (+ 4912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:41.135115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:41.135160 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:41.135789 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:41.137777 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:41.137782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:41.137785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:41.138082 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:41.138854 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:41.138863 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:41.139355 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.139670 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.140261 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8626,"thread_start_us":8399,"threads_started":1}
I20260809 22:46:41.142709 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.109637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.21.27.190:45017)
0809 22:46:41.118581 (+ 8944us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.118588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.118612 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:41.118680 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.118686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.118791 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.118922 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:41.121473 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.122805 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.124926 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.125761 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.125898 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.130868 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:41.130876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:41.130881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:41.130937 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:41.137350 (+ 6413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.138350 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.138356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.138359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.138525 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.139011 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.139016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.139020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.139384 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:41.139649 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.142081 (+ 2432us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.142337 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":8801,"thread_start_us":2473,"threads_started":1}
I20260809 22:46:41.143374 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.108634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:48583 (local address 127.21.27.190:45017)
0809 22:46:41.112341 (+ 3707us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.112347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.115973 (+ 3626us) server_negotiation.cc:407] Connection header received
0809 22:46:41.119237 (+ 3264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.119243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.119330 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.119410 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:41.121676 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.123012 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.125227 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.125992 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.126113 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.130483 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:41.130491 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:41.130496 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:41.130546 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:41.134387 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.137508 (+ 3121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.137512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.137516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.137651 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.138232 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.138237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.138240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.138627 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:41.138777 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.142257 (+ 3480us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.142478 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":3577,"thread_start_us":9649,"threads_started":1}
I20260809 22:46:41.145500 3765 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45017
I20260809 22:46:41.145843 3765 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:41.147117 3765 heartbeater.cc:511] Master 127.21.27.190:45017 requested a full tablet report, sending...
I20260809 22:46:41.148871 3223 ts_manager.cc:194] Registered new tserver with Master: edc16942d89f4788be52919e1eff0231 (127.21.27.131:44707)
I20260809 22:46:41.149894 3223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:48583
I20260809 22:46:41.159909 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:41.199239 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.165289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.21.27.190:45017)
0809 22:46:41.165533 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.165542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.180381 (+ 14839us) server_negotiation.cc:407] Connection header received
0809 22:46:41.180659 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.180665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.180783 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.180888 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:41.183640 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.184814 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.186475 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.187175 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.187319 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.191112 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:41.191119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:41.191124 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:41.191208 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:41.194882 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.195943 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.195948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.195952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.196127 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.196630 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.196635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.196638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.197016 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:41.197271 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.198556 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.198868 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:46:41.206332 3223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
Aug 09 22:46:41 dist-test-slave-chsk krb5kdc[2102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for HTTP/127.21.27.148@KRBTEST.COM
W20260809 22:46:41.216724 3223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:41.279825 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.254236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:34273 (local address 127.0.0.1:60900)
0809 22:46:41.255819 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.255838 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.255945 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.260570 (+ 4625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.260578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.260622 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:41.271676 (+ 11054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.271695 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.273833 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.273838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.276425 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.276442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.278678 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.278685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.278909 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.278971 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.279132 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.279357 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"mutex_wait_us":10525,"spinlock_wait_cycles":896,"thread_start_us":1279,"threads_started":1}
I20260809 22:46:41.280799 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.254973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.21.27.130:34273)
0809 22:46:41.259625 (+ 4652us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.259636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.259670 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:46:41.259781 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.259790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.260076 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.260288 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:41.271947 (+ 11659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.273617 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.276642 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.278135 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.278272 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.278489 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.280315 (+ 1826us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.280515 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4417,"thread_start_us":4571,"threads_started":1}
I20260809 22:46:41.284946 3536 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe8eacd63444553971813d3d4405172 (DEFAULT_TABLE table=test-table [id=45f026a814074753bfcd16cc42a1465d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:41.286574 3536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe8eacd63444553971813d3d4405172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:41.288434 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.245170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42384)
0809 22:46:41.254871 (+ 9701us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.254922 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.255116 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.260575 (+ 5459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.260579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.260629 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:41.261018 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.261040 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.281139 (+ 20099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.281145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.283271 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.283285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.287536 (+ 4251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.287543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.287745 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.287779 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.287849 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.287996 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9383,"mutex_wait_us":3,"spinlock_wait_cycles":3840,"thread_start_us":11039,"threads_started":1}
I20260809 22:46:41.289736 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.262904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:44707 (local address 127.0.0.1:55750)
0809 22:46:41.280951 (+ 18047us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.280976 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.281096 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.282127 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.282132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.282158 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:41.282499 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.282509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.284407 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.284412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.286296 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.286311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.288946 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.288953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.289134 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.289167 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.289266 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.289396 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17927,"thread_start_us":17731,"threads_started":1}
I20260809 22:46:41.290555 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.263610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.21.27.131:44707)
0809 22:46:41.280429 (+ 16819us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.280442 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.281219 (+ 777us) server_negotiation.cc:407] Connection header received
0809 22:46:41.281334 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.281342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.281595 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.281800 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:41.282721 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.284245 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.286477 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.288428 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.288554 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.288784 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.290019 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.290180 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16592,"thread_start_us":16190,"threads_started":1}
I20260809 22:46:41.294586 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.245468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.21.27.129:35777)
0809 22:46:41.259625 (+ 14157us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.259651 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.259680 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:46:41.259781 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.259790 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.260069 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.260290 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:41.261650 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.277685 (+ 16035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.283511 (+ 5826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.284986 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.289041 (+ 4055us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.289300 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.289379 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.289553 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13912,"thread_start_us":15715,"threads_started":1}
I20260809 22:46:41.294361 3695 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe8eacd63444553971813d3d4405172 (DEFAULT_TABLE table=test-table [id=45f026a814074753bfcd16cc42a1465d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:41.295908 3695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe8eacd63444553971813d3d4405172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:41.298408 3375 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe8eacd63444553971813d3d4405172 (DEFAULT_TABLE table=test-table [id=45f026a814074753bfcd16cc42a1465d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:41.301491 3375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe8eacd63444553971813d3d4405172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:41.327051 3789 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Bootstrap starting.
I20260809 22:46:41.332508 3787 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Bootstrap starting.
I20260809 22:46:41.339613 3790 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Bootstrap starting.
I20260809 22:46:41.341683 3789 tablet_bootstrap.cc:654] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:41.346194 3790 tablet_bootstrap.cc:654] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:41.346470 3789 log.cc:824] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:41.347657 3790 log.cc:824] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:41.348469 3787 tablet_bootstrap.cc:654] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:41.349738 3787 log.cc:824] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:41.356382 3789 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: No bootstrap required, opened a new log
I20260809 22:46:41.358131 3787 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: No bootstrap required, opened a new log
I20260809 22:46:41.358496 3787 ts_tablet_manager.cc:1397] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260809 22:46:41.360805 3789 ts_tablet_manager.cc:1397] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.000s
I20260809 22:46:41.377190 3790 tablet_bootstrap.cc:492] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: No bootstrap required, opened a new log
I20260809 22:46:41.382994 3789 raft_consensus.cc:348] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.384727 3787 raft_consensus.cc:348] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.385170 3787 raft_consensus.cc:374] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:41.385272 3787 raft_consensus.cc:729] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc16942d89f4788be52919e1eff0231, State: Initialized, Role: FOLLOWER
I20260809 22:46:41.385977 3790 ts_tablet_manager.cc:1397] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Time spent bootstrapping tablet: real 0.049s user 0.017s sys 0.013s
I20260809 22:46:41.386053 3787 consensus_queue.cc:260] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.387321 3765 heartbeater.cc:503] Master 127.21.27.190:45017 was elected leader, sending a full tablet report...
I20260809 22:46:41.388018 3787 ts_tablet_manager.cc:1428] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Time spent starting tablet: real 0.029s user 0.008s sys 0.005s
I20260809 22:46:41.388019 3789 raft_consensus.cc:374] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:41.388145 3789 raft_consensus.cc:729] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3340ce2bf20041948d4f407dd70b5cf2, State: Initialized, Role: FOLLOWER
I20260809 22:46:41.388839 3789 consensus_queue.cc:260] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.398463 3618 heartbeater.cc:503] Master 127.21.27.190:45017 was elected leader, sending a full tablet report...
I20260809 22:46:41.399478 3789 ts_tablet_manager.cc:1428] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Time spent starting tablet: real 0.038s user 0.022s sys 0.000s
I20260809 22:46:41.404083 3790 raft_consensus.cc:348] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.404649 3790 raft_consensus.cc:374] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:41.404764 3790 raft_consensus.cc:729] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe33b84fc2647be9f675363dddaa5ab, State: Initialized, Role: FOLLOWER
I20260809 22:46:41.405416 3790 consensus_queue.cc:260] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.413826 3790 ts_tablet_manager.cc:1428] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Time spent starting tablet: real 0.028s user 0.009s sys 0.011s
W20260809 22:46:41.414151 3451 tablet.cc:2369] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:41.415094 3449 heartbeater.cc:503] Master 127.21.27.190:45017 was elected leader, sending a full tablet report...
W20260809 22:46:41.473634 3619 tablet.cc:2369] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:41.610946 3766 tablet.cc:2369] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:41.623292 3797 raft_consensus.cc:482] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:41.623623 3797 raft_consensus.cc:504] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.625422 3797 leader_election.cc:288] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3340ce2bf20041948d4f407dd70b5cf2 (127.21.27.130:34273), edc16942d89f4788be52919e1eff0231 (127.21.27.131:44707)
I20260809 22:46:41.633749 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.626161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:33453 (local address 127.21.27.130:34273)
0809 22:46:41.626410 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.626419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.627548 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 22:46:41.627750 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.627762 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.627832 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.627933 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:41.628599 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.629644 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.631945 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.632757 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.632950 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.633023 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.633335 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.633467 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:46:41.633821 3806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.626667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:34273 (local address 127.21.27.129:33453)
0809 22:46:41.627436 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.627451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.627602 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.627950 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.627954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.627981 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:41.628400 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.628411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.629802 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.629807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.631795 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.631806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.632979 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.632985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.633155 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.633185 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.633309 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.633457 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":407,"threads_started":1}
I20260809 22:46:41.633970 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.625968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:49773 (local address 127.21.27.131:44707)
0809 22:46:41.626163 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.626169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.626904 (+ 735us) server_negotiation.cc:407] Connection header received
0809 22:46:41.627177 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.627183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.627263 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.627327 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:41.628240 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.629076 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.631577 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.632786 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.632947 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.633023 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.633580 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.633724 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:46:41.634817 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.625957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:44707 (local address 127.21.27.129:49773)
0809 22:46:41.626765 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.626789 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:41.626961 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:41.627553 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:41.627560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:41.627662 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:41.628071 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.628084 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.629223 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.629229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.631384 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:41.631406 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.633186 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.633193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:41.633378 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.633411 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:46:41.634255 (+ 844us) client_negotiation.cc:240] Negotiation successful
0809 22:46:41.634405 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":5,"spinlock_wait_cycles":25856,"thread_start_us":443,"threads_started":1}
I20260809 22:46:41.634941 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe8eacd63444553971813d3d4405172" candidate_uuid: "dfe33b84fc2647be9f675363dddaa5ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3340ce2bf20041948d4f407dd70b5cf2" is_pre_election: true
I20260809 22:46:41.635650 3565 raft_consensus.cc:2454] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe33b84fc2647be9f675363dddaa5ab in term 0.
I20260809 22:46:41.635949 3717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe8eacd63444553971813d3d4405172" candidate_uuid: "dfe33b84fc2647be9f675363dddaa5ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edc16942d89f4788be52919e1eff0231" is_pre_election: true
I20260809 22:46:41.636430 3327 leader_election.cc:302] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3340ce2bf20041948d4f407dd70b5cf2, dfe33b84fc2647be9f675363dddaa5ab; no voters:
I20260809 22:46:41.636471 3717 raft_consensus.cc:2454] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe33b84fc2647be9f675363dddaa5ab in term 0.
I20260809 22:46:41.636871 3797 raft_consensus.cc:2781] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:41.637033 3797 raft_consensus.cc:482] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:41.637210 3797 raft_consensus.cc:3037] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:41.639611 3797 raft_consensus.cc:504] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.640417 3797 leader_election.cc:288] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [CANDIDATE]: Term 1 election: Requested vote from peers 3340ce2bf20041948d4f407dd70b5cf2 (127.21.27.130:34273), edc16942d89f4788be52919e1eff0231 (127.21.27.131:44707)
I20260809 22:46:41.640933 3565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe8eacd63444553971813d3d4405172" candidate_uuid: "dfe33b84fc2647be9f675363dddaa5ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3340ce2bf20041948d4f407dd70b5cf2"
I20260809 22:46:41.640985 3717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fe8eacd63444553971813d3d4405172" candidate_uuid: "dfe33b84fc2647be9f675363dddaa5ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edc16942d89f4788be52919e1eff0231"
I20260809 22:46:41.641160 3565 raft_consensus.cc:3037] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:41.641230 3717 raft_consensus.cc:3037] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:41.643690 3565 raft_consensus.cc:2454] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe33b84fc2647be9f675363dddaa5ab in term 1.
I20260809 22:46:41.644449 3717 raft_consensus.cc:2454] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe33b84fc2647be9f675363dddaa5ab in term 1.
I20260809 22:46:41.645190 3327 leader_election.cc:302] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3340ce2bf20041948d4f407dd70b5cf2, dfe33b84fc2647be9f675363dddaa5ab; no voters:
I20260809 22:46:41.645680 3797 raft_consensus.cc:2781] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:41.647459 3797 raft_consensus.cc:686] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 1 LEADER]: Becoming Leader. State: Replica: dfe33b84fc2647be9f675363dddaa5ab, State: Running, Role: LEADER
I20260809 22:46:41.648130 3797 consensus_queue.cc:237] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } }
I20260809 22:46:41.653787 3223 catalog_manager.cc:5495] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab reported cstate change: term changed from 0 to 1, leader changed from <none> to dfe33b84fc2647be9f675363dddaa5ab (127.21.27.129). New cstate: current_term: 1 leader_uuid: "dfe33b84fc2647be9f675363dddaa5ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfe33b84fc2647be9f675363dddaa5ab" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 35777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:41.719123 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.712994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.21.27.129:35777)
0809 22:46:41.713215 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.713224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.713554 (+ 330us) server_negotiation.cc:407] Connection header received
0809 22:46:41.713806 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.713811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.713914 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.714138 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:41.714610 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.715430 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.716865 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.717270 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.717417 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.717920 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:41.718488 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:41.718676 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.718729 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.718814 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:46:41.727846 3565 raft_consensus.cc:1261] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 1 FOLLOWER]: Refusing update from remote peer dfe33b84fc2647be9f675363dddaa5ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:41.727843 3717 raft_consensus.cc:1261] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 1 FOLLOWER]: Refusing update from remote peer dfe33b84fc2647be9f675363dddaa5ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:41.729084 3807 consensus_queue.cc:1035] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "edc16942d89f4788be52919e1eff0231" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:41.732295 3807 consensus_queue.cc:1035] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "3340ce2bf20041948d4f407dd70b5cf2" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:41.752786 3813 mvcc.cc:205] Tried to move back new op lower bound from 7316748704665653248 to 7316748704356306944. Current Snapshot: MvccSnapshot[applied={T|T < 7316748704665653248}]
I20260809 22:46:41.776616 3222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:41.776945 3222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:41.781556 3222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fb608ca279440f5a33f46168bf57013: Sending DeleteTablet for 3 replicas of tablet 9fe8eacd63444553971813d3d4405172
I20260809 22:46:41.782974 3375 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe8eacd63444553971813d3d4405172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:41 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:42384
I20260809 22:46:41.782974 3536 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe8eacd63444553971813d3d4405172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:41 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:60900
I20260809 22:46:41.786701 3695 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fe8eacd63444553971813d3d4405172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:41 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:55750
I20260809 22:46:41.788094 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3270
I20260809 22:46:41.812216 3823 tablet_replica.cc:330] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: stopping tablet replica
I20260809 22:46:41.812841 3823 raft_consensus.cc:2229] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:41.813426 3823 raft_consensus.cc:2258] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:41.815768 3821 tablet_replica.cc:330] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: stopping tablet replica
I20260809 22:46:41.816277 3821 raft_consensus.cc:2229] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:41.817070 3822 tablet_replica.cc:330] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: stopping tablet replica
I20260809 22:46:41.817478 3822 raft_consensus.cc:2229] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:41.818037 3822 raft_consensus.cc:2258] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:41.820443 3823 ts_tablet_manager.cc:1905] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:41.830036 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.789088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.21.27.129:35777)
0809 22:46:41.789265 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.789272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.808532 (+ 19260us) server_negotiation.cc:407] Connection header received
0809 22:46:41.808781 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.808787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.808890 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.809009 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:41.811545 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.812714 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.817426 (+ 4712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.818107 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.818213 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.822313 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:41.822331 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:41.822349 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:41.822394 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:41.826424 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.827391 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.827401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.827405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.827594 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.828051 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.828056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.828063 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.828515 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:41.828812 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.829387 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.829649 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:46:41.830006 3821 raft_consensus.cc:2258] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:41.831842 3444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:41.833168 3822 ts_tablet_manager.cc:1905] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:41.839493 3823 ts_tablet_manager.cc:1918] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:41.839809 3823 log.cc:1196] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/9fe8eacd63444553971813d3d4405172
I20260809 22:46:41.840480 3823 ts_tablet_manager.cc:1939] T 9fe8eacd63444553971813d3d4405172 P edc16942d89f4788be52919e1eff0231: Deleting consensus metadata
I20260809 22:46:41.840744 3822 ts_tablet_manager.cc:1918] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:41.840922 3822 log.cc:1196] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/9fe8eacd63444553971813d3d4405172
I20260809 22:46:41.841413 3822 ts_tablet_manager.cc:1939] T 9fe8eacd63444553971813d3d4405172 P 3340ce2bf20041948d4f407dd70b5cf2: Deleting consensus metadata
I20260809 22:46:41.842876 3209 catalog_manager.cc:4841] TS edc16942d89f4788be52919e1eff0231 (127.21.27.131:44707): tablet 9fe8eacd63444553971813d3d4405172 (table test-table [id=45f026a814074753bfcd16cc42a1465d]) successfully deleted
I20260809 22:46:41.843083 3208 catalog_manager.cc:4841] TS 3340ce2bf20041948d4f407dd70b5cf2 (127.21.27.130:34273): tablet 9fe8eacd63444553971813d3d4405172 (table test-table [id=45f026a814074753bfcd16cc42a1465d]) successfully deleted
I20260809 22:46:41.940023 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3270
I20260809 22:46:41.940429 3821 ts_tablet_manager.cc:1905] T 9fe8eacd63444553971813d3d4405172 P dfe33b84fc2647be9f675363dddaa5ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:41.968914 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3454
W20260809 22:46:41.968787 3206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.27.129:35777 (error 108)
I20260809 22:46:41.971580 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.970545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42420)
0809 22:46:41.971298 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:41.971408 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":447,"threads_started":1}
W20260809 22:46:41.971880 3206 catalog_manager.cc:4584] TS dfe33b84fc2647be9f675363dddaa5ab (127.21.27.129:35777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9fe8eacd63444553971813d3d4405172: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
I20260809 22:46:41.983819 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:41.970094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60906 (local address 127.21.27.130:34273)
0809 22:46:41.970316 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:41.970325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:41.970345 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:41.970414 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:41.970421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:41.970501 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:41.970608 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:41.972428 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.973165 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.974682 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:41.975086 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:41.975315 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:41.977780 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:41.977794 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:41.977809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:41.977847 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:41.980875 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.981618 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.981626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.981628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.981783 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:41.982160 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:41.982164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:41.982166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:41.982548 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:41.982838 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:41.983202 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 22:46:41.983457 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:46:41.984580 3611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:42.025578 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.025093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42422)
0809 22:46:42.025327 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.025433 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 22:46:42.026597 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.026245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42430)
0809 22:46:42.026393 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.026467 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:46:42.074535 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3454
I20260809 22:46:42.087193 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.086535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42442)
0809 22:46:42.086815 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.086942 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:46:42.088516 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.088004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42456)
0809 22:46:42.088222 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.088324 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:46:42.095786 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3622
I20260809 22:46:42.110765 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.096638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.21.27.131:44707)
0809 22:46:42.096817 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:42.096822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:42.096842 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:42.096982 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:42.096986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:42.097059 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:42.097141 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:42.098469 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:42.099264 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:42.100659 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:42.101302 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:42.101440 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:42.104049 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:42.104060 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:42.104076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:42.104120 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:42.108012 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:42.108751 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:42.108760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:42.108764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:42.108900 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:42.109238 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:42.109243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:42.109246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:42.109674 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:42.109964 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:42.110168 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 22:46:42.110398 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:46:42.111549 3760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:42.169360 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.168753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42464)
0809 22:46:42.169039 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.169165 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 22:46:42.170528 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:42.170052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:35777 (local address 127.0.0.1:42472)
0809 22:46:42.170278 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:42.170367 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 22:46:42.205644 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3622
I20260809 22:46:42.224993 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3188
I20260809 22:46:42.226202 3263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:42.343441 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3188
I20260809 22:46:42.369549 21614 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 22:46:42.715317 21614 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 22:46:42.715588 21614 mini_ranger.cc:66] Stopping Ranger...
I20260809 22:46:43.060951 21614 mini_ranger.cc:68] Stopped Ranger
2026-08-09 22:46:43.061 UTC [2134] LOG: received smart shutdown request
2026-08-09 22:46:43.064 UTC [2134] LOG: background worker "logical replication launcher" (PID 2144) exited with exit code 1
2026-08-09 22:46:43.064 UTC [2138] LOG: shutting down
2026-08-09 22:46:43.231 UTC [2134] LOG: database system is shut down
2026-08-09T22:46:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54187 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:43 dist-test-slave-chsk krb5kdc[3840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:43 dist-test-slave-chsk krb5kdc[3840](info): set up 2 sockets
Aug 09 22:46:43 dist-test-slave-chsk krb5kdc[3840](info): commencing operation
krb5kdc: starting...
W20260809 22:46:44.527436 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:46:44 dist-test-slave-chsk krb5kdc[3840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:46:44.805016 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:33149
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:45.138417 3854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:45.138703 3854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:45.138897 3854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:45.146090 3854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:45.146214 3854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:45.146298 3854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:45.146402 3854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:45.146484 3854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:45.156313 3854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:33149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:45.157603 3854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:45.158885 3854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:45.170692 3859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:45.170799 3862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:45.170746 3860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:45.171996 3854 server_base.cc:1034] running on GCE node
I20260809 22:46:45.172811 3854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:45.174114 3854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:45.175293 3854 hybrid_clock.cc:648] HybridClock initialized: now 1786315605175234 us; error 51 us; skew 500 ppm
I20260809 22:46:45.175644 3854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:45.178766 3854 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:45.180173 3854 webserver.cc:460] Webserver started at http://127.21.27.190:40857/ using document root <none> and password file <none>
I20260809 22:46:45.180860 3854 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:45.181005 3854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:45.181309 3854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:45.184350 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "7109299d366043f7902b1dc924da8e6a"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "a033b187e058fd7a33f694fabdb7a7a1"
server_key_iv: "6638675ca07df5dc867bc61f0b125af7"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.185168 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "7109299d366043f7902b1dc924da8e6a"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "a033b187e058fd7a33f694fabdb7a7a1"
server_key_iv: "6638675ca07df5dc867bc61f0b125af7"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.190245 3854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 22:46:45.193603 3870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:45.194844 3854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:46:45.195027 3854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "7109299d366043f7902b1dc924da8e6a"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "a033b187e058fd7a33f694fabdb7a7a1"
server_key_iv: "6638675ca07df5dc867bc61f0b125af7"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.195298 3854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:45.205577 3854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:45.209823 3854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:45.210110 3854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:45.229785 3854 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:33149
I20260809 22:46:45.229809 3924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:33149 every 8 connection(s)
I20260809 22:46:45.231223 3854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:45.231602 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3854
I20260809 22:46:45.231848 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:46:45.232823 21614 external_mini_cluster.cc:1428] Setting key 8a199badca72d75019dcbed0979d8d8b
I20260809 22:46:45.235260 3925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.241734 3925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.27.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.189@KRBTEST.COM" created.
I20260809 22:46:45.266136 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.244902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:33149 (local address 127.0.0.1:51640)
0809 22:46:45.245706 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.245742 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.248172 (+ 2430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.249138 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.249147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.249977 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:45.250481 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.250507 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.252202 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.252211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.254150 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.254168 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.255159 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.255169 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.255434 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.256187 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:45.256224 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:45.258862 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:45.262207 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.262222 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.262237 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.262845 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.263263 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.263271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.263278 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.263488 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.264217 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:45.264227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:45.264488 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.264778 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.265358 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":82,"thread_start_us":358,"threads_started":1}
I20260809 22:46:45.266165 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.246245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.21.27.190:33149)
0809 22:46:45.247014 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.247022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.247070 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:46:45.248652 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.248659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.248880 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.249015 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:45.250706 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.252031 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.254368 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.254880 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.255012 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.259108 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:45.259118 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:45.259134 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:45.259210 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:45.262069 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.262966 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.262972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.262974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.263121 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.263598 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.263601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.263603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.264062 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:45.264209 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.264894 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.265161 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":574,"thread_start_us":401,"threads_started":1}
I20260809 22:46:45.269447 3925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:46:45.270681 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.270135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49850)
0809 22:46:45.270325 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.270456 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 22:46:45.270969 3874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
I20260809 22:46:45.272589 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.272247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49864)
0809 22:46:45.272416 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.272479 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 22:46:45.272797 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:37961: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
I20260809 22:46:45.292555 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:37961
--webserver_interface=127.21.27.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:45.322099 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } attempt: 1
I20260809 22:46:45.323319 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.322813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49866)
0809 22:46:45.323003 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.323123 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:46:45.324324 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.323986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49872)
0809 22:46:45.324136 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.324205 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:46:45.324517 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:37961: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
I20260809 22:46:45.392745 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } attempt: 2
I20260809 22:46:45.393961 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.393467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49882)
0809 22:46:45.393726 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.393814 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:46:45.395009 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.394628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49888)
0809 22:46:45.394790 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.394879 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 22:46:45.395294 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:37961: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
I20260809 22:46:45.486606 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } attempt: 3
I20260809 22:46:45.487994 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.487523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49892)
0809 22:46:45.487752 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.487862 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:46:45.489040 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.488699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49894)
0809 22:46:45.488847 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.488913 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:46:45.489215 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:37961: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
W20260809 22:46:45.626153 3932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:45.626499 3932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:45.626662 3932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 22:46:45.632457 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } attempt: 4
I20260809 22:46:45.633767 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.633125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49902)
0809 22:46:45.633479 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.633597 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
W20260809 22:46:45.634282 3932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:45.634416 3932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:45.634521 3932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:45.634616 3932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:45.634693 3932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:45.634964 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.634524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49904)
0809 22:46:45.634707 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.634816 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 22:46:45.635253 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:37961: Network error: Client connection negotiation failed: client connection to 127.21.27.189:37961: connect: Connection refused (error 111)
I20260809 22:46:45.645172 3932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:37961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.27.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:45.646589 3932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:45.647974 3932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:45.659938 3940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:45.659976 3938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:45.660228 3937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:45.660962 3932 server_base.cc:1034] running on GCE node
I20260809 22:46:45.661747 3932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:45.663350 3932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:45.664565 3932 hybrid_clock.cc:648] HybridClock initialized: now 1786315605664510 us; error 47 us; skew 500 ppm
I20260809 22:46:45.664919 3932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:45.673187 3932 init.cc:377] Logged in from keytab as kudu/127.21.27.189@KRBTEST.COM (short username kudu)
I20260809 22:46:45.674770 3932 webserver.cc:460] Webserver started at http://127.21.27.189:34291/ using document root <none> and password file <none>
I20260809 22:46:45.675544 3932 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:45.675693 3932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:45.676005 3932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:45.679093 3932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data/instance:
uuid: "5ab655898f414343991626d1a02a7772"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "b95791550dd0d55a46ed97c5ef727cda"
server_key_iv: "6f3c512f12c941474abfb421e80e1908"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.679977 3932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal/instance:
uuid: "5ab655898f414343991626d1a02a7772"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "b95791550dd0d55a46ed97c5ef727cda"
server_key_iv: "6f3c512f12c941474abfb421e80e1908"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.685091 3932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:46:45.688473 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:45.689687 3932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:45.689864 3932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "5ab655898f414343991626d1a02a7772"
format_stamp: "Formatted at 2026-08-09 22:46:45 on dist-test-slave-chsk"
server_key: "b95791550dd0d55a46ed97c5ef727cda"
server_key_iv: "6f3c512f12c941474abfb421e80e1908"
server_key_version: "encryptionkey@0"
I20260809 22:46:45.690055 3932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:45.712651 3932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:45.717480 3932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:45.717793 3932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:45.740036 3932 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.189:37961
I20260809 22:46:45.740075 4006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.189:37961 every 8 connection(s)
I20260809 22:46:45.741452 3932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
I20260809 22:46:45.745258 4008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:45.749785 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 3932
I20260809 22:46:45.750010 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-1/wal/instance
I20260809 22:46:45.750590 21614 external_mini_cluster.cc:1428] Setting key 937dbb7f27faff706cc7bdefc55856f0
I20260809 22:46:45.753351 4008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.27.188@KRBTEST.COM; defaulting to no policy
I20260809 22:46:45.778118 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.757132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.21.27.190:33149)
0809 22:46:45.757392 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.757402 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.758654 (+ 1252us) server_negotiation.cc:407] Connection header received
0809 22:46:45.760695 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.760704 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.760804 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.760936 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:45.763097 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.763849 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.766343 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.766779 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.766877 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.771460 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:45.771466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:45.771469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:45.771509 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:45.774530 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.775504 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.775512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.775516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.775671 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.776162 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.776166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.776168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.776477 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:45.776659 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.777488 (+ 829us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.777763 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:46:45.779240 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.757559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:33149 (local address 127.0.0.1:51644)
0809 22:46:45.758450 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.758506 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.760397 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.761214 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.761226 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.762047 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:45.762882 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.762909 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.764007 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.764015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.766187 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.766205 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.767018 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.767027 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.767386 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.768179 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:45.768220 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:45.771189 (+ 2969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:45.774733 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.774743 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.774763 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.775362 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.775799 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.775804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.775808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.776012 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.776630 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:45.776639 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:45.777078 (+ 439us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.777372 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.777943 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":397,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.188@KRBTEST.COM" created.
I20260809 22:46:45.781330 4008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:46:45.798924 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.782269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49920)
0809 22:46:45.782505 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.782527 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.782695 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.783630 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.783633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.784036 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:45.784188 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.784196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.786293 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.786297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.787294 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.787307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.788615 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.788620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.788852 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.789572 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:45.789591 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:45.791869 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:45.795658 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.795666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.795673 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.796255 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.796659 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.796667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.796674 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.796870 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.797527 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:45.797535 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:45.797712 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.797884 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.798295 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":39}
I20260809 22:46:45.798929 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.782291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.21.27.189:37961)
0809 22:46:45.783041 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.783060 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.783096 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:46:45.783187 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.783197 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.783453 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.783614 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:45.784366 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.786125 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.787439 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.788426 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.788583 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.792111 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:45.792124 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:45.792138 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:45.792173 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:45.795501 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.796397 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.796404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.796406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.796532 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.797000 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.797003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.797006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.797389 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:45.797574 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.798033 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.798449 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":186,"server-negotiator.queue_time_us":542,"thread_start_us":357,"threads_started":1}
I20260809 22:46:45.801775 4008 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:46:45.803342 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.802658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57394)
0809 22:46:45.802904 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.803073 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260809 22:46:45.803683 3950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:45.805536 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.805042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57404)
0809 22:46:45.805249 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.805369 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 22:46:45.805776 4008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
I20260809 22:46:45.816249 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:36859
--webserver_interface=127.21.27.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:45.855177 4008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 1
I20260809 22:46:45.856482 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.855929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57414)
0809 22:46:45.856151 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.856287 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 22:46:45.857687 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.857247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57416)
0809 22:46:45.857435 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.857530 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 22:46:45.857964 4008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:45.926254 4008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 2
I20260809 22:46:45.927591 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.926969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57432)
0809 22:46:45.927234 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.927353 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:46:45.928812 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.928374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57444)
0809 22:46:45.928547 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.928644 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 22:46:45.929080 4008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:45.934573 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } attempt: 5
Aug 09 22:46:45 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:46:45.949653 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.935493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49928)
0809 22:46:45.935680 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.935698 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:45.935811 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:45.936377 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:45.936385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:45.936812 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:45.937001 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.937010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.938058 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.938064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.938937 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:45.938949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.939821 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.939827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:45.940044 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.940712 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:45.940729 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:45.943034 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:45.946993 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.946999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.947002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.947505 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.947961 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:45.947965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:45.947968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:45.948131 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:45.948870 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:45.948873 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:45.949023 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 22:46:45.949143 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 22:46:45.949387 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:46:45.950063 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.935475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.21.27.189:37961)
0809 22:46:45.936067 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:45.936072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:45.936088 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:46:45.936159 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:45.936162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:45.936222 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:45.936336 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:45.937179 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.937900 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.939100 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:45.939658 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:45.939806 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:45.943280 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:45.943287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:45.943291 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:45.943334 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:45.946803 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.947629 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.947635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.947638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.947803 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:45.948252 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:45.948258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:45.948263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:45.948733 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:45.948910 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:45.949321 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 22:46:45.949622 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463}
I20260809 22:46:45.951330 3925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:46:45.952533 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.952117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57458)
0809 22:46:45.952301 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.952394 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 22:46:45.953557 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:45.953215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57472)
0809 22:46:45.953369 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:45.953454 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:46:45.953794 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:46.005039 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 1
I20260809 22:46:46.006210 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.005735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57484)
0809 22:46:46.005934 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.006043 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 22:46:46.007277 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.006883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57494)
0809 22:46:46.007028 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.007105 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:46:46.007493 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:46.020334 4008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 3
I20260809 22:46:46.021732 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.021073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57506)
0809 22:46:46.021342 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.021479 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 22:46:46.022994 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.022401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57522)
0809 22:46:46.022693 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.022812 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260809 22:46:46.023331 4008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:46.075730 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 2
I20260809 22:46:46.076907 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.076397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57524)
0809 22:46:46.076651 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.076737 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:46:46.078083 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.077633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57538)
0809 22:46:46.077807 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.077916 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 22:46:46.078346 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:46.166608 4008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 4
W20260809 22:46:46.166627 4015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:46.166982 4015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:46.167207 4015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 22:46:46.167997 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.167433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57552)
0809 22:46:46.167696 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.167801 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:46:46.169013 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.168654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57556)
0809 22:46:46.168804 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.168886 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:46:46.169221 4008 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
W20260809 22:46:46.175226 4015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:46.175371 4015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:46.175457 4015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:46.175539 4015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:46.175599 4015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:46.184584 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 3
I20260809 22:46:46.185823 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.185269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57558)
0809 22:46:46.185500 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.185614 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:46:46.186451 4015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:33149,127.21.27.189:37961,127.21.27.188:36859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:36859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.27.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:46.186949 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.186516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57568)
0809 22:46:46.186726 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.186819 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 22:46:46.187290 3925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:36859: Network error: Client connection negotiation failed: client connection to 127.21.27.188:36859: connect: Connection refused (error 111)
I20260809 22:46:46.188009 4015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:46.189297 4015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:46.201488 4021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:46.201442 4024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:46.201568 4020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:46.207193 4015 server_base.cc:1034] running on GCE node
I20260809 22:46:46.208160 4015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:46.209709 4015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:46.210949 4015 hybrid_clock.cc:648] HybridClock initialized: now 1786315606210908 us; error 52 us; skew 500 ppm
I20260809 22:46:46.211380 4015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:46.214591 4015 init.cc:377] Logged in from keytab as kudu/127.21.27.188@KRBTEST.COM (short username kudu)
I20260809 22:46:46.216286 4015 webserver.cc:460] Webserver started at http://127.21.27.188:34237/ using document root <none> and password file <none>
I20260809 22:46:46.217034 4015 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:46.217177 4015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:46.217519 4015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:46.220651 4015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data/instance:
uuid: "022dfc93c03143cb8552ed39766ef251"
format_stamp: "Formatted at 2026-08-09 22:46:46 on dist-test-slave-chsk"
server_key: "9968a2e88170f188892664bcb1447a46"
server_key_iv: "dd9ea9cd50d3cde24bea483dba41bdb2"
server_key_version: "encryptionkey@0"
I20260809 22:46:46.221520 4015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal/instance:
uuid: "022dfc93c03143cb8552ed39766ef251"
format_stamp: "Formatted at 2026-08-09 22:46:46 on dist-test-slave-chsk"
server_key: "9968a2e88170f188892664bcb1447a46"
server_key_iv: "dd9ea9cd50d3cde24bea483dba41bdb2"
server_key_version: "encryptionkey@0"
I20260809 22:46:46.226958 4015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 22:46:46.230573 4032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:46.232146 4015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:46.232332 4015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "022dfc93c03143cb8552ed39766ef251"
format_stamp: "Formatted at 2026-08-09 22:46:46 on dist-test-slave-chsk"
server_key: "9968a2e88170f188892664bcb1447a46"
server_key_iv: "dd9ea9cd50d3cde24bea483dba41bdb2"
server_key_version: "encryptionkey@0"
I20260809 22:46:46.232530 4015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:46.264648 4015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:46.268965 4015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:46.269289 4015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:46.288838 4015 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.188:36859
I20260809 22:46:46.288872 4086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.188:36859 every 8 connection(s)
I20260809 22:46:46.290445 4015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
I20260809 22:46:46.294556 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4015
I20260809 22:46:46.294590 4087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:46.294790 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786315394803050-21614-0/minicluster-data/master-2/wal/instance
I20260809 22:46:46.295387 21614 external_mini_cluster.cc:1428] Setting key b34288c2ab5adba2a30c4e969b6e506c
I20260809 22:46:46.304904 4087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:46.320178 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.297331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.21.27.188:36859)
0809 22:46:46.298290 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.298303 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.298343 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:46:46.299976 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.299999 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.300444 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.300799 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.302774 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.304104 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.306148 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.306844 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.306992 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.311939 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.311946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.311956 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.311995 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.315714 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.316458 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.316467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.316475 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.316693 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.317050 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.317055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.317059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.317677 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.317953 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.318350 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.318667 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":496,"threads_started":1}
I20260809 22:46:46.325661 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.307853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:33149 (local address 127.0.0.1:51660)
0809 22:46:46.308616 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.308677 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.308842 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.309958 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.309968 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.310637 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.310967 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.310976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.312334 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.312339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.314191 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.314212 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.315104 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.315112 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.315348 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.316033 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.316050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.318622 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.322301 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.322315 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.322321 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.322833 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.323809 (+ 976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.323814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.323818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.323964 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.324547 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.324554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.324731 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.324908 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.325232 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":544,"threads_started":1}
I20260809 22:46:46.325700 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.307919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.21.27.190:33149)
0809 22:46:46.309618 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.309622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.309639 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:46:46.309696 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.309700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.309757 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.309844 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.311135 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.312172 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.314468 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.314905 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.315004 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.318850 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.318857 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.318861 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.318902 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.322162 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.323446 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.323451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.323455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.323639 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.324099 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.324106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.324109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.324409 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.324584 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.325019 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.325239 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1604,"thread_start_us":1470,"threads_started":1}
I20260809 22:46:46.327044 4087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:46:46.340242 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.323309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.21.27.190:33149)
0809 22:46:46.323990 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.323994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.324013 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:46.324051 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.324054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.324104 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.324226 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.325706 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.326831 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.328686 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.329101 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.329252 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.334033 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.334039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.334042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.334075 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.337345 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.338048 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.338053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.338057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.338198 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.338592 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.338597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.338601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.338945 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.339248 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.339610 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.339783 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":362,"threads_started":1}
I20260809 22:46:46.341533 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.327899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:37961 (local address 127.0.0.1:49944)
0809 22:46:46.328121 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.328138 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.328251 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.328540 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.328545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.328917 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.329050 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.329058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.330244 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.330249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.331780 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.331797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.332586 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.332590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.332758 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.333333 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.333351 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.335680 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.338924 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.338930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.338934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.339469 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.339897 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.339902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.339906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.340094 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.340699 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.340703 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.340842 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.340950 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.341153 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 22:46:46.341730 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.327885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.21.27.189:37961)
0809 22:46:46.328049 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.328054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.328214 (+ 160us) server_negotiation.cc:407] Connection header received
0809 22:46:46.328353 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.328356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.328413 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.328488 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.329203 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.330104 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.331951 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.332414 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.332588 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.335902 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.335909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.335916 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.335961 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.338794 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.339592 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.339597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.339601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.339763 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.340222 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.340226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.340228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.340572 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.340726 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.341118 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.341385 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:46:46.342760 4087 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315606, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:46:46.355823 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.343615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57590)
0809 22:46:46.343756 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.343770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.343842 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.344239 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.344243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.344609 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.344734 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.344742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.345627 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.345634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.346455 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.346465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.347195 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.347201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.347371 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.348046 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.348063 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.349962 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.353146 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.353149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.353152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.353475 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.353839 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.353845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.353849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.353984 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.354735 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.354743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.354881 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.355020 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.355372 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:46:46.355844 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.343859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.21.27.188:36859)
0809 22:46:46.344006 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.344029 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.344042 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:46:46.344075 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.344078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.344123 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.344225 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.344865 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.345504 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.346594 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.347010 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.347152 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.350138 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.350142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.350145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.350172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.353013 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.353602 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.353610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.353614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.353734 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.354274 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.354279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.354283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.354593 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.354762 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.355111 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.355532 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":60}
I20260809 22:46:46.357159 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.342298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.21.27.189:37961)
0809 22:46:46.342457 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.342462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.342591 (+ 129us) server_negotiation.cc:407] Connection header received
0809 22:46:46.342778 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.342783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.342848 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.342946 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.344495 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.345239 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.346507 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.346933 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.347066 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.351453 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.351458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.351461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.351492 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.354058 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.354739 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.354745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.354750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.354901 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.355357 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.355363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.355368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.355616 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.356348 (+ 732us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.356689 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.356868 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:46:46.360852 4087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: Bootstrap starting.
I20260809 22:46:46.361675 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:46.363992 4087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.364511 3925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 4
I20260809 22:46:46.365268 4087 log.cc:824] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:46.369294 4087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: No bootstrap required, opened a new log
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:46:46.384218 4087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.384706 4087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.384842 4087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022dfc93c03143cb8552ed39766ef251, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.388593 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.365091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57600)
0809 22:46:46.365459 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.365480 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.365607 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.366325 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.366329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.366918 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.367171 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.367186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.369069 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.369075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.370507 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.370524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.377013 (+ 6489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.377030 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.377215 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.378008 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.378038 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.380885 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.385459 (+ 4574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.385465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.385468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.386041 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.386543 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.386550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.386555 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.386749 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.387471 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.387477 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.387656 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.387814 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.388141 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260809 22:46:46.390592 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.365839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57600 (local address 127.21.27.188:36859)
0809 22:46:46.366055 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.366059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.366073 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:46:46.366116 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.366119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.366169 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.366297 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.367352 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.368435 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.375904 (+ 7469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.376723 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.377614 (+ 891us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.381107 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.381114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.381119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.381160 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.385295 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.386210 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.386215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.386225 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.386397 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.386912 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.386918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.386923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.387301 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.389793 (+ 2492us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.389947 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.390197 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 22:46:46.392518 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.366608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.21.27.190:33149)
0809 22:46:46.366789 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.366795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.368789 (+ 1994us) server_negotiation.cc:407] Connection header received
0809 22:46:46.374688 (+ 5899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.374694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.374772 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.374841 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.377330 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.378387 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.380727 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.381372 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.381469 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.385700 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.385706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.385710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.385746 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.389249 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.390138 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.390144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.390147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.390284 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.390750 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.390755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.390758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.391111 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.391393 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.391944 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.392186 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:46:46.395917 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.367701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.21.27.189:37961)
0809 22:46:46.367886 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.367893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.368460 (+ 567us) server_negotiation.cc:407] Connection header received
0809 22:46:46.368741 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.368746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.368835 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.368928 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.371974 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.373440 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.376930 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.377573 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.377679 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.387865 (+ 10186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.387875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.387881 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.387931 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.391770 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.392692 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.392699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.392703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.392930 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.394034 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.394039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.394043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.394439 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.394694 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.395313 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.395562 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:46:46.393814 4087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.397271 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.368797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.21.27.188:36859)
0809 22:46:46.378311 (+ 9514us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.378317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.378339 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:46:46.378397 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.378402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.378473 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.378581 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.381646 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.383029 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.385222 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.385986 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.386117 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.389681 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.389688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.389695 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.389743 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.393370 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.394323 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.394328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.394331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.394472 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.394978 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.394983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.394992 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.395433 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.395666 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.396573 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.396801 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9373,"spinlock_wait_cycles":32000,"thread_start_us":8020,"threads_started":1}
I20260809 22:46:46.397316 3925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: Bootstrap starting.
I20260809 22:46:46.397596 4105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.398142 4105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.399999 4087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:46.401309 3925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.402563 3925 log.cc:824] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:46.405882 3925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: No bootstrap required, opened a new log
I20260809 22:46:46.416221 4105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:46.416572 4105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.408758 4015 master_runner.cc:186] Error getting master registration for 127.21.27.190:33149: OK, instance_id { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" instance_seqno: 1786315605224145 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:46:46.417750 4015 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:46:46.418226 4105 leader_election.cc:288] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7109299d366043f7902b1dc924da8e6a (127.21.27.190:33149), 5ab655898f414343991626d1a02a7772 (127.21.27.189:37961)
W20260809 22:46:46.419322 4118 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:46:46.419447 4118 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:46:46.419674 3925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.419955 3925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.420089 3925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7109299d366043f7902b1dc924da8e6a, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.420843 3925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.421795 4120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.422271 4120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.422992 3925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:46.423036 3899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dfc93c03143cb8552ed39766ef251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7109299d366043f7902b1dc924da8e6a" is_pre_election: true
I20260809 22:46:46.423678 3899 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 022dfc93c03143cb8552ed39766ef251 in term 0.
I20260809 22:46:46.424325 4035 leader_election.cc:302] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dfc93c03143cb8552ed39766ef251, 7109299d366043f7902b1dc924da8e6a; no voters:
I20260809 22:46:46.424746 4105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:46.424885 4105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:46.424995 4105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.428556 4105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.431897 3899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dfc93c03143cb8552ed39766ef251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7109299d366043f7902b1dc924da8e6a"
I20260809 22:46:46.432111 3899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.435532 3899 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 022dfc93c03143cb8552ed39766ef251 in term 1.
I20260809 22:46:46.435906 4105 leader_election.cc:288] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [CANDIDATE]: Term 1 election: Requested vote from peers 7109299d366043f7902b1dc924da8e6a (127.21.27.190:33149), 5ab655898f414343991626d1a02a7772 (127.21.27.189:37961)
I20260809 22:46:46.436373 4035 leader_election.cc:302] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 022dfc93c03143cb8552ed39766ef251, 7109299d366043f7902b1dc924da8e6a; no voters:
I20260809 22:46:46.436695 4105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:46.437522 4105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [term 1 LEADER]: Becoming Leader. State: Replica: 022dfc93c03143cb8552ed39766ef251, State: Running, Role: LEADER
I20260809 22:46:46.440055 4105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.447503 4128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dfc93c03143cb8552ed39766ef251. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.448060 4128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:46.449394 4135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:46.451810 4135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 22:46:46.463258 3877 tablet.cc:2369] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:46.452344 3854 master_runner.cc:186] Error getting master registration for 127.21.27.189:37961: OK, instance_id { permanent_uuid: "5ab655898f414343991626d1a02a7772" instance_seqno: 1786315605732420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:46:46.465251 3854 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 22:46:46.467320 4134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:46:46.467494 4134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:46:46.468459 4008 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } attempt: 5
I20260809 22:46:46.480799 3899 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [term 1 FOLLOWER]: Refusing update from remote peer 022dfc93c03143cb8552ed39766ef251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Aug 09 22:46:46 dist-test-slave-chsk krb5kdc[3840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315605, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:46:46.481932 4128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.499503 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.470160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.21.27.188:36859)
0809 22:46:46.470322 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:46.470327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:46.470346 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:46:46.470401 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:46.470406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:46.470480 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:46.470578 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:46.471914 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.473088 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.475402 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.476116 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.476346 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.482586 (+ 6240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:46.482594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:46.482598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:46.482640 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:46.491903 (+ 9263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.493628 (+ 1725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.493638 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.493643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.494024 (+ 381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:46.494836 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:46.494842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:46.494873 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:46.495653 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:46.496497 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:46.497033 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 22:46:46.497940 (+ 907us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:46:46.499819 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:46.469019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:36859 (local address 127.0.0.1:57610)
0809 22:46:46.469553 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:46.469574 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:46.469692 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:46.470861 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:46.470866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:46.471481 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:46.471698 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.471711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.473699 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.473704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.475193 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:46.475210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:46.476576 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:46.476582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:46.476833 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:46.477780 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:46.477806 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:46.482291 (+ 4485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:46.492198 (+ 9907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.492205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.492210 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.493224 (+ 1014us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.494289 (+ 1065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:46.494294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:46.494298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:46.494589 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:46.495865 (+ 1276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:46.495870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:46.496170 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 22:46:46.496705 (+ 535us) client_negotiation.cc:240] Negotiation successful
0809 22:46:46.497906 (+ 1201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401}
I20260809 22:46:46.506649 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: Bootstrap starting.
I20260809 22:46:46.511857 4008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:46.513372 4008 log.cc:824] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:46.514569 4120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dfc93c03143cb8552ed39766ef251. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.514935 4120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.518687 4137 mvcc.cc:205] Tried to move back new op lower bound from 7316748724067811328 to 7316748723983605760. Current Snapshot: MvccSnapshot[applied={T|T < 7316748724067811328}]
I20260809 22:46:46.520399 4138 mvcc.cc:205] Tried to move back new op lower bound from 7316748724067811328 to 7316748723983605760. Current Snapshot: MvccSnapshot[applied={T|T < 7316748724067811328}]
I20260809 22:46:46.536438 4105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.537125 4105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:46.538009 4120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.538272 4120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.542241 4135 catalog_manager.cc:1347] Generated new cluster ID: 19a81a3f939a49eb89e9c0fb4384d5ae
I20260809 22:46:46.542371 4135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:46.544799 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: No bootstrap required, opened a new log
I20260809 22:46:46.560240 4008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.561414 4008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:46.561769 4008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab655898f414343991626d1a02a7772, State: Initialized, Role: FOLLOWER
I20260809 22:46:46.565028 4008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } }
I20260809 22:46:46.571828 4135 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:46.575095 4135 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:46.579710 4143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.580168 4143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.582820 3978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dfc93c03143cb8552ed39766ef251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab655898f414343991626d1a02a7772"
I20260809 22:46:46.583248 3977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:46.583770 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "022dfc93c03143cb8552ed39766ef251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab655898f414343991626d1a02a7772" is_pre_election: true
I20260809 22:46:46.585384 4008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:46.587049 3977 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [term 1 FOLLOWER]: Refusing update from remote peer 022dfc93c03143cb8552ed39766ef251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:46.587947 4128 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:46.601769 4135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251: Generated new TSK 0
I20260809 22:46:46.602917 4135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:46.680718 4105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.681078 4105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 022dfc93c03143cb8552ed39766ef251 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:46.682873 4143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 022dfc93c03143cb8552ed39766ef251. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.683667 4143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.684598 4150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "022dfc93c03143cb8552ed39766ef251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7109299d366043f7902b1dc924da8e6a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33149 } } peers { permanent_uuid: "5ab655898f414343991626d1a02a7772" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 37961 } } peers { permanent_uuid: "022dfc93c03143cb8552ed39766ef251" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 36859 } } }
I20260809 22:46:46.685007 4150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:46:46.710316 4166 catalog_manager.cc:1259] Loaded cluster ID: 19a81a3f939a49eb89e9c0fb4384d5ae
I20260809 22:46:46.710444 4166 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: loading cluster ID for follower catalog manager: success
I20260809 22:46:46.714123 4166 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: acquiring CA information for follower catalog manager: success
I20260809 22:46:46.717001 4166 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ab655898f414343991626d1a02a7772: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:46:47.469822 4134 catalog_manager.cc:1259] Loaded cluster ID: 19a81a3f939a49eb89e9c0fb4384d5ae
I20260809 22:46:47.469942 4134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: loading cluster ID for follower catalog manager: success
I20260809 22:46:47.472383 4134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: acquiring CA information for follower catalog manager: success
I20260809 22:46:47.474116 4134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7109299d366043f7902b1dc924da8e6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:46:47.750886 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3854
I20260809 22:46:47.752417 3920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.862524 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3854
I20260809 22:46:47.887108 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 3932
I20260809 22:46:47.888269 4001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:47.996941 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 3932
I20260809 22:46:48.020540 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4015
I20260809 22:46:48.021647 4081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:48.130565 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4015
2026-08-09T22:46:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4771 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:48 dist-test-slave-chsk krb5kdc[4179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:48 dist-test-slave-chsk krb5kdc[4179](info): set up 2 sockets
Aug 09 22:46:48 dist-test-slave-chsk krb5kdc[4179](info): commencing operation
krb5kdc: starting...
W20260809 22:46:49.284941 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.006s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:46:49 dist-test-slave-chsk krb5kdc[4179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:46:49.560671 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:46235
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:42919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:46235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:49.898435 4194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:49.898754 4194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:49.898849 4194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:49.906421 4194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:49.906558 4194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:49.906605 4194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:49.906670 4194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:49.906716 4194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:49.917089 4194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:42919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:46235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:46235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:49.918429 4194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:49.919785 4194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:49.931584 4199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:49.931785 4200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:49.932502 4202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:49.932835 4194 server_base.cc:1034] running on GCE node
I20260809 22:46:49.933894 4194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:49.935536 4194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:49.936780 4194 hybrid_clock.cc:648] HybridClock initialized: now 1786315609936741 us; error 57 us; skew 500 ppm
I20260809 22:46:49.937160 4194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:49 dist-test-slave-chsk krb5kdc[4179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315609, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:49.940335 4194 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:49.942041 4194 webserver.cc:460] Webserver started at http://127.21.27.190:33801/ using document root <none> and password file <none>
I20260809 22:46:49.942745 4194 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:49.942903 4194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:49.943282 4194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:49.946266 4194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "3231a58b8ac54b2c95b15be5268adb4a"
format_stamp: "Formatted at 2026-08-09 22:46:49 on dist-test-slave-chsk"
server_key: "75e167d4ecf0aac65e18b88c65ff659d"
server_key_iv: "66a537920907565b0487138836800371"
server_key_version: "encryptionkey@0"
I20260809 22:46:49.947048 4194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "3231a58b8ac54b2c95b15be5268adb4a"
format_stamp: "Formatted at 2026-08-09 22:46:49 on dist-test-slave-chsk"
server_key: "75e167d4ecf0aac65e18b88c65ff659d"
server_key_iv: "66a537920907565b0487138836800371"
server_key_version: "encryptionkey@0"
I20260809 22:46:49.952410 4194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:46:49.955757 4211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:49.957032 4194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:49.957217 4194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "3231a58b8ac54b2c95b15be5268adb4a"
format_stamp: "Formatted at 2026-08-09 22:46:49 on dist-test-slave-chsk"
server_key: "75e167d4ecf0aac65e18b88c65ff659d"
server_key_iv: "66a537920907565b0487138836800371"
server_key_version: "encryptionkey@0"
I20260809 22:46:49.957413 4194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:49.988298 4194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:49.992338 4194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:49.992655 4194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:50.012018 4194 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:46235
I20260809 22:46:50.012061 4263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:46235 every 8 connection(s)
I20260809 22:46:50.013357 4194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:50.016913 4264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:50.017539 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4194
I20260809 22:46:50.017789 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:46:50.018690 21614 external_mini_cluster.cc:1428] Setting key 5fcb4dfec6da80ec743292a64fd54fb7
I20260809 22:46:50.031172 4264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a: Bootstrap starting.
Aug 09 22:46:50 dist-test-slave-chsk krb5kdc[4179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:50.034832 4264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:50.036031 4264 log.cc:824] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:50.038810 4264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a: No bootstrap required, opened a new log
I20260809 22:46:50.042888 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:50.020588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.21.27.190:46235)
0809 22:46:50.021354 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:50.021365 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:50.021405 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:46:50.023501 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:50.023521 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:50.023887 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:50.024199 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:50.026068 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:50.027389 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:50.029229 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:50.029853 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:50.029972 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:50.034983 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:50.034995 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:50.035017 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:50.035057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:50.038317 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:50.039050 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:50.039058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:50.039065 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:50.039309 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:50.039693 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:50.039698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:50.039702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:50.040400 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:50.040703 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:50.041103 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 22:46:50.041557 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":348,"threads_started":1}
I20260809 22:46:50.046977 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:46:50.048388 4264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } }
I20260809 22:46:50.048810 4264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:50.048913 4264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3231a58b8ac54b2c95b15be5268adb4a, State: Initialized, Role: FOLLOWER
I20260809 22:46:50.049511 4264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } }
I20260809 22:46:50.049821 4264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:50.049970 4264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:50.050186 4264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:50.052524 4264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } }
I20260809 22:46:50.052990 4264 leader_election.cc:302] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3231a58b8ac54b2c95b15be5268adb4a; no voters:
I20260809 22:46:50.053583 4264 leader_election.cc:288] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:50.053797 4269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:50.054610 4269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [term 1 LEADER]: Becoming Leader. State: Replica: 3231a58b8ac54b2c95b15be5268adb4a, State: Running, Role: LEADER
I20260809 22:46:50.055130 4269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } }
I20260809 22:46:50.055693 4264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:50.058429 4271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3231a58b8ac54b2c95b15be5268adb4a. Latest consensus state: current_term: 1 leader_uuid: "3231a58b8ac54b2c95b15be5268adb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } } }
I20260809 22:46:50.058846 4271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:50.058535 4270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3231a58b8ac54b2c95b15be5268adb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3231a58b8ac54b2c95b15be5268adb4a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 46235 } } }
I20260809 22:46:50.059062 4270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:50.060751 4276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:50.066237 4276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:50.074461 4276 catalog_manager.cc:1347] Generated new cluster ID: 3d10fa272e0e4bb0b61c376b562b62d0
I20260809 22:46:50.074656 4276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:50.086928 4276 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:50.088248 4276 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:50.098117 4276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3231a58b8ac54b2c95b15be5268adb4a: Generated new TSK 0
I20260809 22:46:50.098981 4276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:46:50.108140 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4194
I20260809 22:46:50.109365 4259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:50.201458 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4194
2026-08-09T22:46:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2067 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:46:50 dist-test-slave-chsk krb5kdc[4292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:46:50 dist-test-slave-chsk krb5kdc[4292](info): set up 2 sockets
Aug 09 22:46:50 dist-test-slave-chsk krb5kdc[4292](info): commencing operation
krb5kdc: starting...
W20260809 22:46:51.371513 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:46:51 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:46:51.655277 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:51.997733 4307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:51.998114 4307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:51.998286 4307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:52.005976 4307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:52.006156 4307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:52.006275 4307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:52.006392 4307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:52.006474 4307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:52.017036 4307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:52.018476 4307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:52.019958 4307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:52.032320 4313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.032277 4316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.032318 4314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:52.033604 4307 server_base.cc:1034] running on GCE node
I20260809 22:46:52.034503 4307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:52.036037 4307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:52.037286 4307 hybrid_clock.cc:648] HybridClock initialized: now 1786315612037232 us; error 38 us; skew 500 ppm
I20260809 22:46:52.037640 4307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:52 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:52.044179 4307 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:52.045841 4307 webserver.cc:460] Webserver started at http://127.21.27.190:45981/ using document root <none> and password file <none>
I20260809 22:46:52.046535 4307 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:52.046695 4307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:52.047026 4307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:52.050108 4307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "31043e5525c14155b07927952a0d74c3"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "f7e95af540fd28b8cdacd30fa490cd08"
server_key_iv: "556ac4f5ca7e9e212ce14d5f81d687b7"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.050917 4307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "31043e5525c14155b07927952a0d74c3"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "f7e95af540fd28b8cdacd30fa490cd08"
server_key_iv: "556ac4f5ca7e9e212ce14d5f81d687b7"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.056461 4307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:46:52.059974 4324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.061427 4307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:52.061563 4307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "31043e5525c14155b07927952a0d74c3"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "f7e95af540fd28b8cdacd30fa490cd08"
server_key_iv: "556ac4f5ca7e9e212ce14d5f81d687b7"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.061731 4307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:52.082734 4307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:52.086944 4307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:52.087278 4307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:52.106297 4307 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40159
I20260809 22:46:52.106343 4378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40159 every 8 connection(s)
I20260809 22:46:52.107769 4307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:52.111452 4379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:52.112461 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4307
I20260809 22:46:52.112700 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:46:52.113389 21614 external_mini_cluster.cc:1428] Setting key ddc370df6ad70292e786f9258ebae722
I20260809 22:46:52.125773 4379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap starting.
Aug 09 22:46:52 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:52.129122 4379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:52.130311 4379 log.cc:824] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:52.133536 4379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: No bootstrap required, opened a new log
I20260809 22:46:52.137178 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:52.115528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.21.27.190:40159)
0809 22:46:52.116346 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:52.116361 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:52.116414 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:46:52.118071 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:52.118097 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:52.118591 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:52.119032 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:52.120425 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.122453 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.123782 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.124847 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.125017 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.129798 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:52.129806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:52.129820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:52.129860 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:52.133030 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.133852 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.133859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.133864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.134028 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.134465 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.134468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.134470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.134955 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:52.135255 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:52.135721 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 22:46:52.136142 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":328,"threads_started":1}
I20260809 22:46:52.142592 4379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:52.143105 4379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:52.143258 4379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Initialized, Role: FOLLOWER
I20260809 22:46:52.143919 4379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:52.144172 4379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:52.144317 4379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:52.144488 4379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:52.147797 4379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:52.148386 4379 leader_election.cc:302] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31043e5525c14155b07927952a0d74c3; no voters:
I20260809 22:46:52.149544 4379 leader_election.cc:288] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:46:52.149791 4384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:52.150739 4384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 LEADER]: Becoming Leader. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Running, Role: LEADER
I20260809 22:46:52.151538 4384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:52.152442 4379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:52.154578 4386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31043e5525c14155b07927952a0d74c3. Latest consensus state: current_term: 1 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:52.154938 4386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:52.154663 4385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:52.155112 4385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:52.156809 4390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:52.162292 4390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:52.171535 4390 catalog_manager.cc:1347] Generated new cluster ID: 4baf5afe14b74cef880aad255d05fc7a
I20260809 22:46:52.171661 4390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:52.189320 4390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:46:52.190659 4390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:52.197898 4390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Generated new TSK 0
I20260809 22:46:52.198714 4390 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:46:52.275092 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:52.641885 4405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:52.642184 4405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:52.642390 4405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:52.649739 4405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:52.649865 4405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:52.650046 4405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:46:52.660104 4405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:52.661397 4405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:52.662777 4405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:52.676141 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.676266 4413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:52.676079 4410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:52.677117 4405 server_base.cc:1034] running on GCE node
I20260809 22:46:52.677734 4405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:52.678709 4405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:52.679925 4405 hybrid_clock.cc:648] HybridClock initialized: now 1786315612679858 us; error 44 us; skew 500 ppm
I20260809 22:46:52.680250 4405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:52 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:52.683531 4405 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:46:52.685091 4405 webserver.cc:460] Webserver started at http://127.21.27.129:44635/ using document root <none> and password file <none>
I20260809 22:46:52.685802 4405 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:52.685935 4405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:52.686259 4405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:52.689144 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "f0ff2376ab0345d1af69705ac16d7e87"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "59e0a7a7419932f4da63b5e4eeb53a20"
server_key_iv: "b1c5564a053aab1b6c80c7039fbe1df2"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.689976 4405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "f0ff2376ab0345d1af69705ac16d7e87"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "59e0a7a7419932f4da63b5e4eeb53a20"
server_key_iv: "b1c5564a053aab1b6c80c7039fbe1df2"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.694870 4405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:46:52.698345 4421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.699659 4405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:52.699798 4405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "f0ff2376ab0345d1af69705ac16d7e87"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "59e0a7a7419932f4da63b5e4eeb53a20"
server_key_iv: "b1c5564a053aab1b6c80c7039fbe1df2"
server_key_version: "encryptionkey@0"
I20260809 22:46:52.700026 4405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:52.713865 4405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:52.718050 4405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:52.718308 4405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:52.719213 4405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:52.720466 4405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:52.720551 4405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.720645 4405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:52.720692 4405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:52.754423 4405 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:32793
I20260809 22:46:52.754473 4542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:32793 every 8 connection(s)
I20260809 22:46:52.755746 4405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:52.756489 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4405
I20260809 22:46:52.756724 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:46:52.757622 21614 external_mini_cluster.cc:1428] Setting key 73ca8d8d6bb318def0499fcec49f100a
Aug 09 22:46:52 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315612, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:52.780227 4545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:52.758566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:46443 (local address 127.21.27.190:40159)
0809 22:46:52.759049 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:52.759053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:52.760269 (+ 1216us) server_negotiation.cc:407] Connection header received
0809 22:46:52.763115 (+ 2846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:52.763119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:52.763244 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:52.763337 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:52.765430 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.766342 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.768854 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.769235 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.769333 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.773310 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:52.773320 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:52.773327 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:52.773373 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:52.776952 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.777625 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.777633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.777639 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.777786 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:52.778220 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:52.778227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:52.778233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:52.778636 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:52.778796 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:52.779520 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 22:46:52.779763 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":271,"threads_started":1}
I20260809 22:46:52.780778 4546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:52.759115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:46443)
0809 22:46:52.760098 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:52.760153 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:52.762832 (+ 2679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:52.763593 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:52.763607 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:52.764427 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:52.765263 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:52.765290 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.766498 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.766506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:52.768706 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:52.768724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:52.769460 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:52.769469 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:52.769810 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:52.770531 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:52.770565 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:52.773082 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:52.777100 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:52.777106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:52.777117 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:52.777496 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:52.777888 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:52.777891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:52.777894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:52.778062 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:52.778758 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:52.778763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:52.779049 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 22:46:52.779328 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 22:46:52.779709 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":387,"threads_started":1}
I20260809 22:46:52.782395 4543 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:52.782716 4543 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:52.783651 4543 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:52.785851 4341 ts_manager.cc:194] Registered new tserver with Master: f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129:32793)
I20260809 22:46:52.787354 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:46443
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:46:52.820513 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:53.183251 4549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:53.183828 4549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:53.184082 4549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:53.195711 4549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:53.195935 4549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:53.196187 4549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:46:53.212158 4549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:53.214079 4549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:53.216418 4549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:53.237151 4558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.237255 4556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.237265 4555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:53.238494 4549 server_base.cc:1034] running on GCE node
I20260809 22:46:53.239473 4549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:53.240816 4549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:53.242092 4549 hybrid_clock.cc:648] HybridClock initialized: now 1786315613242007 us; error 77 us; skew 500 ppm
I20260809 22:46:53.242488 4549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:53 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:53.247628 4549 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:46:53.249938 4549 webserver.cc:460] Webserver started at http://127.21.27.130:36551/ using document root <none> and password file <none>
I20260809 22:46:53.251052 4549 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:53.251289 4549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:53.251816 4549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:53.256945 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "11d0389561a54330ae02b65758312fc1"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "5d3b41168a2f2ea316cdc02eee16f68f"
server_key_iv: "a05f33ca331dbcabf6df6df3c9a4fa8b"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.258186 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "11d0389561a54330ae02b65758312fc1"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "5d3b41168a2f2ea316cdc02eee16f68f"
server_key_iv: "a05f33ca331dbcabf6df6df3c9a4fa8b"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.267488 4549 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.009s
I20260809 22:46:53.273483 4565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.275596 4549 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:46:53.275854 4549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "11d0389561a54330ae02b65758312fc1"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "5d3b41168a2f2ea316cdc02eee16f68f"
server_key_iv: "a05f33ca331dbcabf6df6df3c9a4fa8b"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.276175 4549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:53.312282 4549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:53.319463 4549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:53.320010 4549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:53.321759 4549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:53.324087 4549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:53.324254 4549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.324479 4549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:53.324599 4549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.378568 4549 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:39679
I20260809 22:46:53.378626 4686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:39679 every 8 connection(s)
I20260809 22:46:53.380772 4549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:53.387890 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4549
I20260809 22:46:53.388185 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:46:53.388942 21614 external_mini_cluster.cc:1428] Setting key 77116b3ca00504893ce7ea04c43cdca5
Aug 09 22:46:53 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:53.418329 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.384898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:33449 (local address 127.21.27.190:40159)
0809 22:46:53.385680 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:53.385684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:53.386899 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 22:46:53.390111 (+ 3212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:53.390119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:53.390307 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:53.390518 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:53.393630 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.395704 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.397839 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.399399 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.399656 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.405383 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:53.405403 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:53.405422 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:53.405549 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:53.412011 (+ 6462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.413781 (+ 1770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.413785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.413787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.414076 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:53.414913 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:53.414917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:53.414919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:53.415423 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:53.415597 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:53.417604 (+ 2007us) server_negotiation.cc:299] Negotiation successful
0809 22:46:53.417893 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":418,"threads_started":1}
I20260809 22:46:53.421401 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:53.385570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:33449)
0809 22:46:53.386661 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:53.386731 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:53.389581 (+ 2850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:53.390987 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:53.391011 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:53.392356 (+ 1345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:53.393363 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.393398 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.395894 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.395897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.397637 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:53.397650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:53.399701 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:53.399707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:53.400015 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:53.401090 (+ 1075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:53.401125 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:53.404958 (+ 3833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:53.412266 (+ 7308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.412301 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.412342 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.413540 (+ 1198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.414255 (+ 715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:53.414274 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:53.414292 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:53.414729 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:53.415630 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:53.415654 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:53.416699 (+ 1045us) client_negotiation.cc:769] Sending connection context
0809 22:46:53.417370 (+ 671us) client_negotiation.cc:240] Negotiation successful
0809 22:46:53.418447 (+ 1077us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":446,"threads_started":1}
I20260809 22:46:53.424063 4687 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:53.424621 4687 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:53.426329 4687 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:53.429565 4341 ts_manager.cc:194] Registered new tserver with Master: 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
I20260809 22:46:53.431047 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:33449
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:46:53.490938 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:53.790258 4543 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
W20260809 22:46:53.916582 4694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:53.916864 4694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:53.916960 4694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:53.924640 4694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:53.924772 4694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:53.924991 4694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:46:53.934778 4694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:53.936070 4694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:53.937384 4694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:53.951898 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.951876 4702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:53.951956 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:53.953054 4694 server_base.cc:1034] running on GCE node
I20260809 22:46:53.953809 4694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:53.954754 4694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:53.955953 4694 hybrid_clock.cc:648] HybridClock initialized: now 1786315613955906 us; error 40 us; skew 500 ppm
I20260809 22:46:53.956271 4694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:53 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:53.959372 4694 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:46:53.960819 4694 webserver.cc:460] Webserver started at http://127.21.27.131:43763/ using document root <none> and password file <none>
I20260809 22:46:53.961504 4694 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:53.961651 4694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:53.961905 4694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:46:53.964967 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "67f19388df5ef35269f9bc4b83ff5344"
server_key_iv: "95d9201841eaf8ac8d295aa1db7c284f"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.965780 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "67f19388df5ef35269f9bc4b83ff5344"
server_key_iv: "95d9201841eaf8ac8d295aa1db7c284f"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.970741 4694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:46:53.974113 4710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:53.975364 4694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:53.975555 4694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "67f19388df5ef35269f9bc4b83ff5344"
server_key_iv: "95d9201841eaf8ac8d295aa1db7c284f"
server_key_version: "encryptionkey@0"
I20260809 22:46:53.975747 4694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:54.004371 4694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:54.008224 4694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:54.008533 4694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:54.009480 4694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:54.010823 4694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:54.010926 4694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:54.011078 4694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:54.011175 4694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:54.044845 4694 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:43269
I20260809 22:46:54.044893 4834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:43269 every 8 connection(s)
I20260809 22:46:54.046103 4694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:54.051368 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4694
I20260809 22:46:54.051566 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:46:54.052071 21614 external_mini_cluster.cc:1428] Setting key 4ddbb9a2f574d97843d39661a9d5796e
Aug 09 22:46:54 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315613, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:54.066361 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.048651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:54715 (local address 127.21.27.190:40159)
0809 22:46:54.049147 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.049153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.050028 (+ 875us) server_negotiation.cc:407] Connection header received
0809 22:46:54.052041 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.052049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.052158 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.052292 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.054293 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.055339 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.056934 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.057355 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.057445 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.060610 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.060614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.060617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.060646 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.063585 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.064198 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.064202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.064204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.064316 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.064706 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.064709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.064711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.064954 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.065102 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.065714 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.065920 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":220,"threads_started":1}
I20260809 22:46:54.067449 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.049172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:54715)
0809 22:46:54.049865 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.049909 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.051789 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.052614 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.052625 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.053393 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:54.054125 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.054148 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.055488 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.055492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.056813 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.056824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.057525 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.057529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.057708 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.058311 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:54.058332 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:54.060434 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:54.063702 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:54.063707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:54.063717 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:54.064095 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:54.064404 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:54.064407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:54.064409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:54.064610 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:54.065072 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:54.065077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:54.065330 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.065575 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.066115 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":272,"threads_started":1}
I20260809 22:46:54.068792 4835 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:54.069036 4835 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:54.069638 4835 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:54.070906 4341 ts_manager.cc:194] Registered new tserver with Master: ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269)
I20260809 22:46:54.071777 4341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:54715
I20260809 22:46:54.080070 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:54.097221 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.084887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.21.27.190:40159)
0809 22:46:54.085046 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.085051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.085548 (+ 497us) server_negotiation.cc:407] Connection header received
0809 22:46:54.085690 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.085693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.085764 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.085844 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.087360 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.088064 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.089335 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.089735 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.089824 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.092243 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.092248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.092251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.092281 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.094963 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.095630 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.095633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.095636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.095754 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.096067 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.096070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.096072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.096380 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.096564 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.096830 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.096992 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:46:54.101047 4341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:54.108261 4341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:54.131711 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.122993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.21.27.131:43269)
0809 22:46:54.123842 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.123849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.123881 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:46:54.123973 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.123982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.124187 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.124920 (+ 733us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:54.125127 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.126153 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.128999 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.130459 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.130596 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.130852 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.131133 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.131335 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":498,"threads_started":1}
I20260809 22:46:54.131945 4849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.120226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:44344)
0809 22:46:54.121081 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.121132 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.121307 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.123039 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.123050 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.123093 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:54.124022 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.124043 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.126685 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.126689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.129006 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.129021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.131026 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.131034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.131252 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.131293 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.131388 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.131523 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":47,"thread_start_us":450,"threads_started":1}
I20260809 22:46:54.132737 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.122373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43269 (local address 127.0.0.1:56176)
0809 22:46:54.122918 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.122935 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.123037 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.124394 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.124400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.124424 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:54.124777 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.124786 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.126287 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.126294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.128849 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.128860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.130656 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.130662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.130848 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.130907 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.131048 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.132384 (+ 1336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":110,"thread_start_us":777,"threads_started":1}
I20260809 22:46:54.132814 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.120554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.21.27.129:32793)
0809 22:46:54.121980 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.121988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.122028 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:46:54.122137 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.122148 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.122440 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.122680 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:54.124331 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.126506 (+ 2175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.129171 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.130599 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.130723 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.130929 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.132369 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.132538 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1258,"thread_start_us":1069,"threads_started":1}
I20260809 22:46:54.134096 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.121651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39679 (local address 127.0.0.1:49676)
0809 22:46:54.122311 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.122327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.122435 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.127312 (+ 4877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.127317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.127346 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:54.128113 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.128127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.129715 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.129720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.131387 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.131401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.133350 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.133357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.133514 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.133555 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.133650 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.133776 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":378,"threads_started":1}
I20260809 22:46:54.134371 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.121916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.21.27.130:39679)
0809 22:46:54.126747 (+ 4831us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.126754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.126778 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:46:54.126864 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.126872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.127081 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.127836 (+ 755us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:54.128324 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.129575 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.131575 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.133099 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.133237 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.133418 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.133875 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.134043 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4663,"thread_start_us":4538,"threads_started":1}
I20260809 22:46:54.136667 4763 tablet_service.cc:1467] Processing CreateTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe (DEFAULT_TABLE table=test-table [id=8156e039b6af4295a90a05caf24893f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:54.136675 4474 tablet_service.cc:1467] Processing CreateTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe (DEFAULT_TABLE table=test-table [id=8156e039b6af4295a90a05caf24893f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:54.137329 4620 tablet_service.cc:1467] Processing CreateTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe (DEFAULT_TABLE table=test-table [id=8156e039b6af4295a90a05caf24893f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:54.138172 4474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78ea616a4fc64c1c96774c9ea3f87ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:54.138190 4763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78ea616a4fc64c1c96774c9ea3f87ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:54.138461 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78ea616a4fc64c1c96774c9ea3f87ffe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:54.149771 4855 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Bootstrap starting.
I20260809 22:46:54.152550 4856 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Bootstrap starting.
I20260809 22:46:54.153717 4858 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Bootstrap starting.
I20260809 22:46:54.154805 4855 tablet_bootstrap.cc:654] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:54.156093 4855 log.cc:824] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:54.156447 4856 tablet_bootstrap.cc:654] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:54.156877 4858 tablet_bootstrap.cc:654] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:54.157648 4856 log.cc:824] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:54.157984 4858 log.cc:824] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:54.159493 4855 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: No bootstrap required, opened a new log
I20260809 22:46:54.159765 4855 ts_tablet_manager.cc:1397] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260809 22:46:54.160583 4856 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: No bootstrap required, opened a new log
I20260809 22:46:54.160852 4856 ts_tablet_manager.cc:1397] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:46:54.161046 4858 tablet_bootstrap.cc:492] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: No bootstrap required, opened a new log
I20260809 22:46:54.161305 4858 ts_tablet_manager.cc:1397] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:46:54.167799 4855 raft_consensus.cc:348] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.168179 4855 raft_consensus.cc:374] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:54.168308 4855 raft_consensus.cc:729] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d0389561a54330ae02b65758312fc1, State: Initialized, Role: FOLLOWER
I20260809 22:46:54.168915 4855 consensus_queue.cc:260] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.170081 4687 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
I20260809 22:46:54.170769 4855 ts_tablet_manager.cc:1428] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260809 22:46:54.171761 4858 raft_consensus.cc:348] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.171725 4856 raft_consensus.cc:348] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.172152 4856 raft_consensus.cc:374] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:54.172168 4858 raft_consensus.cc:374] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:54.172252 4856 raft_consensus.cc:729] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ff2376ab0345d1af69705ac16d7e87, State: Initialized, Role: FOLLOWER
I20260809 22:46:54.172253 4858 raft_consensus.cc:729] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0afcabd9dc4ac9ae37496b6224d127, State: Initialized, Role: FOLLOWER
I20260809 22:46:54.172848 4856 consensus_queue.cc:260] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.172993 4858 consensus_queue.cc:260] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.174857 4856 ts_tablet_manager.cc:1428] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 22:46:54.175567 4835 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
I20260809 22:46:54.176017 4858 ts_tablet_manager.cc:1428] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260809 22:46:54.258733 4544 tablet.cc:2369] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:54.297899 4836 tablet.cc:2369] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:54.384104 4688 tablet.cc:2369] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:54.481010 4863 raft_consensus.cc:482] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:54.481343 4863 raft_consensus.cc:504] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.483489 4863 leader_election.cc:288] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679), ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269)
I20260809 22:46:54.490701 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.483823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:46957 (local address 127.21.27.130:39679)
0809 22:46:54.484064 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.484073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.484586 (+ 513us) server_negotiation.cc:407] Connection header received
0809 22:46:54.484808 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.484815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.484904 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.485001 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:54.485745 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.486753 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.488873 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.489719 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.489835 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.489884 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.490296 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.490435 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:46:54.490748 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.484733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:37319 (local address 127.21.27.131:43269)
0809 22:46:54.484945 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.484951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.485367 (+ 416us) server_negotiation.cc:407] Connection header received
0809 22:46:54.485452 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.485458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.485536 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.485629 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:54.486334 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.487228 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.489032 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.489847 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.490006 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.490079 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.490382 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.490518 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:46:54.490819 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.483815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39679 (local address 127.21.27.129:46957)
0809 22:46:54.484448 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.484465 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.484660 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.485041 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.485049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.485091 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:54.485484 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.485497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.486887 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.486892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.488712 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.488724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.489935 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.489943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.490140 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.490165 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.490241 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.490372 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":90,"thread_start_us":367,"threads_started":1}
I20260809 22:46:54.491273 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.484613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43269 (local address 127.21.27.129:37319)
0809 22:46:54.485247 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:54.485266 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:54.485364 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:54.485739 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:54.485743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:54.485769 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:54.486104 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.486116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.487398 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.487403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.488880 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:54.488894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.490080 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.490084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:54.490193 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.490236 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 22:46:54.490332 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:46:54.490448 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":282,"threads_started":1}
I20260809 22:46:54.491933 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78ea616a4fc64c1c96774c9ea3f87ffe" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11d0389561a54330ae02b65758312fc1" is_pre_election: true
I20260809 22:46:54.492266 4785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78ea616a4fc64c1c96774c9ea3f87ffe" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" is_pre_election: true
I20260809 22:46:54.492578 4640 raft_consensus.cc:2454] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 0.
I20260809 22:46:54.492841 4785 raft_consensus.cc:2454] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 0.
I20260809 22:46:54.493218 4422 leader_election.cc:302] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11d0389561a54330ae02b65758312fc1, f0ff2376ab0345d1af69705ac16d7e87; no voters:
I20260809 22:46:54.493607 4863 raft_consensus.cc:2781] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:54.493773 4863 raft_consensus.cc:482] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:54.493893 4863 raft_consensus.cc:3037] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:54.496178 4863 raft_consensus.cc:504] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.496757 4863 leader_election.cc:288] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 election: Requested vote from peers 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679), ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269)
I20260809 22:46:54.497097 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78ea616a4fc64c1c96774c9ea3f87ffe" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11d0389561a54330ae02b65758312fc1"
I20260809 22:46:54.497198 4785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78ea616a4fc64c1c96774c9ea3f87ffe" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
I20260809 22:46:54.497288 4640 raft_consensus.cc:3037] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:54.497412 4785 raft_consensus.cc:3037] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:54.499552 4640 raft_consensus.cc:2454] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 1.
I20260809 22:46:54.499629 4785 raft_consensus.cc:2454] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 1.
I20260809 22:46:54.499959 4422 leader_election.cc:302] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11d0389561a54330ae02b65758312fc1, f0ff2376ab0345d1af69705ac16d7e87; no voters:
I20260809 22:46:54.500236 4863 raft_consensus.cc:2781] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:54.500677 4863 raft_consensus.cc:686] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 1 LEADER]: Becoming Leader. State: Replica: f0ff2376ab0345d1af69705ac16d7e87, State: Running, Role: LEADER
I20260809 22:46:54.501153 4863 consensus_queue.cc:237] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } }
I20260809 22:46:54.505857 4341 catalog_manager.cc:5495] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "f0ff2376ab0345d1af69705ac16d7e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } health_report { overall_health: UNKNOWN } } }
I20260809 22:46:54.588801 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.583278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44350 (local address 127.21.27.129:32793)
0809 22:46:54.583551 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.583558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.583657 (+ 99us) server_negotiation.cc:407] Connection header received
0809 22:46:54.583844 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.583848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.583954 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.584092 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:54.584415 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.585123 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.586543 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.586934 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.587032 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.587590 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:54.588102 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:54.588217 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.588400 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.588470 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:46:54.597434 4785 raft_consensus.cc:1261] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Refusing update from remote peer f0ff2376ab0345d1af69705ac16d7e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:54.597442 4640 raft_consensus.cc:1261] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Refusing update from remote peer f0ff2376ab0345d1af69705ac16d7e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:54.598480 4863 consensus_queue.cc:1035] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:54.598876 4867 consensus_queue.cc:1035] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:54.624773 4872 mvcc.cc:205] Tried to move back new op lower bound from 7316748757380034560 to 7316748757000523776. Current Snapshot: MvccSnapshot[applied={T|T < 7316748757380034560}]
I20260809 22:46:54.631954 4341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:54.632277 4341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:54.636229 4341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Sending DeleteTablet for 3 replicas of tablet 78ea616a4fc64c1c96774c9ea3f87ffe
I20260809 22:46:54.637583 4474 tablet_service.cc:1514] Processing DeleteTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:54 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:44344
I20260809 22:46:54.637755 4763 tablet_service.cc:1514] Processing DeleteTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:54 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:56176
I20260809 22:46:54.638597 4881 tablet_replica.cc:330] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: stopping tablet replica
I20260809 22:46:54.638584 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4405
I20260809 22:46:54.638679 4620 tablet_service.cc:1514] Processing DeleteTablet for tablet 78ea616a4fc64c1c96774c9ea3f87ffe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:54 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:49676
I20260809 22:46:54.639317 4881 raft_consensus.cc:2229] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:54.640244 4881 raft_consensus.cc:2258] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:54.643049 4881 ts_tablet_manager.cc:1905] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:54.651681 4881 ts_tablet_manager.cc:1918] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:54.652045 4881 log.cc:1196] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/78ea616a4fc64c1c96774c9ea3f87ffe
I20260809 22:46:54.652837 4881 ts_tablet_manager.cc:1939] T 78ea616a4fc64c1c96774c9ea3f87ffe P f0ff2376ab0345d1af69705ac16d7e87: Deleting consensus metadata
I20260809 22:46:54.653435 4882 tablet_replica.cc:330] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: stopping tablet replica
I20260809 22:46:54.653859 4882 raft_consensus.cc:2229] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:54.654419 4882 raft_consensus.cc:2258] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:54.655190 4325 catalog_manager.cc:4841] TS f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129:32793): tablet 78ea616a4fc64c1c96774c9ea3f87ffe (table test-table [id=8156e039b6af4295a90a05caf24893f4]) successfully deleted
I20260809 22:46:54.657698 4882 ts_tablet_manager.cc:1905] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:54.661083 4883 tablet_replica.cc:330] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: stopping tablet replica
I20260809 22:46:54.661712 4883 raft_consensus.cc:2229] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:54.662349 4883 raft_consensus.cc:2258] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:54.664084 4882 ts_tablet_manager.cc:1918] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:54.664276 4882 log.cc:1196] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/78ea616a4fc64c1c96774c9ea3f87ffe
I20260809 22:46:54.664726 4883 ts_tablet_manager.cc:1905] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:54.664870 4882 ts_tablet_manager.cc:1939] T 78ea616a4fc64c1c96774c9ea3f87ffe P ed0afcabd9dc4ac9ae37496b6224d127: Deleting consensus metadata
I20260809 22:46:54.666872 4325 catalog_manager.cc:4841] TS ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): tablet 78ea616a4fc64c1c96774c9ea3f87ffe (table test-table [id=8156e039b6af4295a90a05caf24893f4]) successfully deleted
Aug 09 22:46:54 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:46:54.672214 4883 ts_tablet_manager.cc:1918] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:54.672408 4883 log.cc:1196] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/78ea616a4fc64c1c96774c9ea3f87ffe
I20260809 22:46:54.672951 4883 ts_tablet_manager.cc:1939] T 78ea616a4fc64c1c96774c9ea3f87ffe P 11d0389561a54330ae02b65758312fc1: Deleting consensus metadata
I20260809 22:46:54.675029 4850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.640073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.21.27.129:32793)
0809 22:46:54.640313 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.640323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.650163 (+ 9840us) server_negotiation.cc:407] Connection header received
0809 22:46:54.650777 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.650787 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.650909 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.651000 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.655949 (+ 4949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.657177 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.659442 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.661818 (+ 2376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.661979 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.668224 (+ 6245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.668238 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.668252 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.668308 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.671673 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.672525 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.672535 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.672539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.672709 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.673178 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.673181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.673183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.673575 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.673921 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.674256 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.674518 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:46:54.675484 4325 catalog_manager.cc:4841] TS 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): tablet 78ea616a4fc64c1c96774c9ea3f87ffe (table test-table [id=8156e039b6af4295a90a05caf24893f4]) successfully deleted
I20260809 22:46:54.675819 4538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.776333 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4405
I20260809 22:46:54.797406 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4549
Aug 09 22:46:54 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:46:54.812975 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.798193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.21.27.130:39679)
0809 22:46:54.798367 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.798371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.798423 (+ 52us) server_negotiation.cc:407] Connection header received
0809 22:46:54.798573 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.798578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.798641 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.798727 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.800518 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.801239 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.802672 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.803049 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.803233 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.807562 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.807576 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.807591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.807630 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.810458 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.811214 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.811221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.811224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.811359 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.811714 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.811717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.811719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.812082 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.812304 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.812555 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.812721 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:46:54.813653 4682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:54.904951 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4549
I20260809 22:46:54.924410 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4694
Aug 09 22:46:54 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:46:54.940316 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:54.925326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.21.27.131:43269)
0809 22:46:54.925520 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:54.925524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:54.925544 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:46:54.925744 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:54.925748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:54.925817 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:54.925921 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:54.927269 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.928439 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.929767 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:54.930427 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:54.930555 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:54.934952 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:54.934969 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:54.934985 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:54.935046 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:54.937699 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.938396 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.938402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.938404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.938543 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:54.938869 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:54.938872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:54.938874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:54.939339 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:54.939562 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:54.939871 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 22:46:54.940052 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:46:54.940999 4829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:55.030458 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4694
I20260809 22:46:55.049652 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4307
I20260809 22:46:55.050889 4373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:55.163100 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4307
I20260809 22:46:55.184545 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--webserver_interface=127.21.27.190
--webserver_port=45981
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:55.520160 4889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:55.520475 4889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:55.520663 4889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:55.528026 4889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:55.528151 4889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:55.528263 4889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:55.528353 4889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:55.528432 4889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:55.538714 4889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=45981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:55.540146 4889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:55.541535 4889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:55.567932 4895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.567983 4899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:55.567999 4896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:55.568706 4889 server_base.cc:1034] running on GCE node
I20260809 22:46:55.569635 4889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:55.571341 4889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:55.572577 4889 hybrid_clock.cc:648] HybridClock initialized: now 1786315615572524 us; error 63 us; skew 500 ppm
I20260809 22:46:55.572999 4889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:55 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315615, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:55.576673 4889 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:55.578393 4889 webserver.cc:460] Webserver started at http://127.21.27.190:45981/ using document root <none> and password file <none>
I20260809 22:46:55.579258 4889 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:55.579434 4889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:55.585644 4889 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 22:46:55.588771 4907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:55.590173 4889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:55.590371 4889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "31043e5525c14155b07927952a0d74c3"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "f7e95af540fd28b8cdacd30fa490cd08"
server_key_iv: "556ac4f5ca7e9e212ce14d5f81d687b7"
server_key_version: "encryptionkey@0"
I20260809 22:46:55.591578 4889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:55.610050 4889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:55.615046 4889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:55.615444 4889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:55.635993 4889 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40159
I20260809 22:46:55.636015 4962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40159 every 8 connection(s)
I20260809 22:46:55.637681 4889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:55.641283 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4889
I20260809 22:46:55.642185 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:32793
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=44635
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:55.645207 4963 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:55.648675 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap starting.
I20260809 22:46:55.676224 4963 log.cc:824] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:55.692670 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:55.693392 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap complete.
I20260809 22:46:55.707880 4963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:55.709290 4963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Initialized, Role: FOLLOWER
I20260809 22:46:55.710060 4963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:55.710363 4963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:55.710505 4963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:55.710695 4963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:46:55.717571 4963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:55.718381 4963 leader_election.cc:302] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31043e5525c14155b07927952a0d74c3; no voters:
I20260809 22:46:55.719074 4963 leader_election.cc:288] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:46:55.719316 4967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:46:55.720724 4967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 LEADER]: Becoming Leader. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Running, Role: LEADER
I20260809 22:46:55.721415 4963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:55.721288 4967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:55.724053 4970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31043e5525c14155b07927952a0d74c3. Latest consensus state: current_term: 2 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:55.724530 4970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:55.725178 4968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:55.725528 4968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:55.737968 4975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:55.748023 4975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:55.749531 4975 catalog_manager.cc:1259] Loaded cluster ID: 4baf5afe14b74cef880aad255d05fc7a
I20260809 22:46:55.749650 4975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:46:55.753037 4975 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 22:46:55.771828 4975 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 22:46:55.792918 4975 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:46:55.794795 4975 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Loaded TSK: 0
I20260809 22:46:55.796989 4975 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:46:56.031044 4965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:56.031395 4965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:56.031560 4965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:56.038940 4965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:56.039104 4965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:56.039304 4965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:46:56.049175 4965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:32793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=44635
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:56.050483 4965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:56.051844 4965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:56.066617 4991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:56.066577 4992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:56.066756 4994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:56.068136 4965 server_base.cc:1034] running on GCE node
I20260809 22:46:56.068898 4965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:56.070145 4965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:56.071400 4965 hybrid_clock.cc:648] HybridClock initialized: now 1786315616071333 us; error 52 us; skew 500 ppm
I20260809 22:46:56.071784 4965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:56 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:56.076124 4965 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:46:56.077980 4965 webserver.cc:460] Webserver started at http://127.21.27.129:44635/ using document root <none> and password file <none>
I20260809 22:46:56.078907 4965 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:56.079087 4965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:56.086553 4965 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:46:56.090075 5001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.091813 4965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:56.092015 4965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "f0ff2376ab0345d1af69705ac16d7e87"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "59e0a7a7419932f4da63b5e4eeb53a20"
server_key_iv: "b1c5564a053aab1b6c80c7039fbe1df2"
server_key_version: "encryptionkey@0"
I20260809 22:46:56.093443 4965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:56.118057 4965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:56.123826 4965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:56.124166 4965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:56.125406 4965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:56.127049 4965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:56.127183 4965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.127298 4965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:56.127378 4965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.162483 4965 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:32793
I20260809 22:46:56.162572 5125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:32793 every 8 connection(s)
I20260809 22:46:56.164276 4965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:56.165860 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 4965
I20260809 22:46:56.166610 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:39679
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=36551
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:56 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:56.197774 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.167650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:48219)
0809 22:46:56.174225 (+ 6575us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.174279 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.177013 (+ 2734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.179217 (+ 2204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.179230 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.179859 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.180383 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.180401 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.182153 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.182157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.183548 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.183561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.184567 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.184575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.184862 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.185576 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.185600 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:56.188171 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:56.193325 (+ 5154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:56.193332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:56.193343 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:56.193791 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:56.194342 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:56.194345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:56.194348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:56.194533 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:56.195690 (+ 1157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:56.195695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:56.196047 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 22:46:56.196345 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 22:46:56.196803 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6231,"thread_start_us":6001,"threads_started":1}
I20260809 22:46:56.198200 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.167368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48219 (local address 127.21.27.190:40159)
0809 22:46:56.177886 (+ 10518us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.177901 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.177956 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:46:56.178287 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.178298 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.178635 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.178998 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.180639 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.181945 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.183695 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.184313 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.184552 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.188559 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:56.188572 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:56.188596 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:56.188663 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:56.193141 (+ 4478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.193955 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.193967 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.193975 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.194208 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.194681 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.194687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.194694 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.195554 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:56.195782 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:56.196626 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 22:46:56.197101 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10261,"thread_start_us":10101,"threads_started":1}
I20260809 22:46:56.201035 5128 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:56.201355 5128 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:56.202224 5128 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:56.204424 4924 ts_manager.cc:194] Registered new tserver with Master: f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129:32793)
I20260809 22:46:56.205906 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:48219
W20260809 22:46:56.504770 5132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:56.505156 5132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:56.505336 5132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:56.512471 5132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:56.512621 5132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:56.512810 5132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:46:56.522464 5132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:39679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=36551
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:56.523772 5132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:56.525118 5132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:56.538064 5137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:56.538292 5140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:56.538095 5138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:56.538821 5132 server_base.cc:1034] running on GCE node
I20260809 22:46:56.539536 5132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:56.540510 5132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:56.541699 5132 hybrid_clock.cc:648] HybridClock initialized: now 1786315616541644 us; error 50 us; skew 500 ppm
I20260809 22:46:56.542022 5132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:56 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:56.545199 5132 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:46:56.546568 5132 webserver.cc:460] Webserver started at http://127.21.27.130:36551/ using document root <none> and password file <none>
I20260809 22:46:56.547343 5132 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:56.547513 5132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:56.552659 5132 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:46:56.555450 5147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.556778 5132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:56.556964 5132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "11d0389561a54330ae02b65758312fc1"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "5d3b41168a2f2ea316cdc02eee16f68f"
server_key_iv: "a05f33ca331dbcabf6df6df3c9a4fa8b"
server_key_version: "encryptionkey@0"
I20260809 22:46:56.558022 5132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:56.577270 5132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:56.581789 5132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:56.582084 5132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:56.583276 5132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:56.584617 5132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:56.584751 5132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.584890 5132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:56.584981 5132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:56.619108 5132 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:39679
I20260809 22:46:56.619195 5269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:39679 every 8 connection(s)
I20260809 22:46:56.620718 5132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:56.626338 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5132
I20260809 22:46:56.627208 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:43269
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=43763
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:46:56 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:56.644505 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.623521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:45675 (local address 127.21.27.190:40159)
0809 22:46:56.623720 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:56.623725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:56.624790 (+ 1065us) server_negotiation.cc:407] Connection header received
0809 22:46:56.627000 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:56.627004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:56.627090 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:56.627252 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:56.629347 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.630038 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.632613 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.633045 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.633146 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.638033 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:56.638038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:56.638041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:56.638094 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:56.641271 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.642102 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.642110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.642115 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.642250 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:56.642690 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:56.642695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:56.642699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:56.642987 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:56.643149 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:56.643885 (+ 736us) server_negotiation.cc:299] Negotiation successful
0809 22:46:56.644121 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:46:56.645665 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:56.623952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:45675)
0809 22:46:56.624633 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:56.624679 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:56.626674 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:56.627492 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:56.627506 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:56.628343 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:56.629143 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.629183 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.630217 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.630226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.632376 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:56.632394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:56.633311 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:56.633321 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:56.633652 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:56.634403 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:56.634445 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:56.637728 (+ 3283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:56.641412 (+ 3684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:56.641421 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:56.641436 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:56.641933 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:56.642367 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:56.642372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:56.642376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:56.642572 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:56.643120 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:56.643127 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:56.643495 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 22:46:56.643784 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 22:46:56.644302 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":260,"threads_started":1}
I20260809 22:46:56.647192 5270 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:56.647505 5270 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:56.648252 5270 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:56.649699 4924 ts_manager.cc:194] Registered new tserver with Master: 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
I20260809 22:46:56.650590 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:45675
W20260809 22:46:56.975714 5273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:56.975991 5273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:56.976086 5273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:56.984357 5273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:56.984477 5273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:56.984602 5273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:46:56.994431 5273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:43269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=43763
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:56.995875 5273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:56.997259 5273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:57.011169 5278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:57.011210 5281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:57.011248 5279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:57.012122 5273 server_base.cc:1034] running on GCE node
I20260809 22:46:57.012773 5273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:57.013754 5273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:57.014987 5273 hybrid_clock.cc:648] HybridClock initialized: now 1786315617014950 us; error 52 us; skew 500 ppm
I20260809 22:46:57.015370 5273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:57 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:57.018553 5273 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:46:57.020145 5273 webserver.cc:460] Webserver started at http://127.21.27.131:43763/ using document root <none> and password file <none>
I20260809 22:46:57.020880 5273 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:57.021049 5273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:57.026762 5273 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 22:46:57.029672 5289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:57.031118 5273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:46:57.031381 5273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "67f19388df5ef35269f9bc4b83ff5344"
server_key_iv: "95d9201841eaf8ac8d295aa1db7c284f"
server_key_version: "encryptionkey@0"
I20260809 22:46:57.032459 5273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:57.058367 5273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:57.062966 5273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:57.063313 5273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:57.064337 5273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:57.065691 5273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:57.065783 5273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:57.065945 5273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:57.066030 5273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:57.100332 5273 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:43269
I20260809 22:46:57.100412 5413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:43269 every 8 connection(s)
I20260809 22:46:57.101868 5273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:57.110610 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5273
Aug 09 22:46:57 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:46:57.124142 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.104670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:47695 (local address 127.21.27.190:40159)
0809 22:46:57.104901 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.104907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.106157 (+ 1250us) server_negotiation.cc:407] Connection header received
0809 22:46:57.107960 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.107964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.108039 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.108133 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.109604 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.110338 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.112536 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.113239 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.113368 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.117027 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:57.117032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:57.117036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:57.117068 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:57.120184 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.121230 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.121238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.121243 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.121394 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.121930 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.121941 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.121945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.122386 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:57.122570 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.123444 (+ 874us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.123748 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:46:57.125165 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.105254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:47695)
0809 22:46:57.105978 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.106021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.107699 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.108352 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.108360 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.108890 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:46:57.109450 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.109467 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.110512 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.110518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.112387 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.112400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.113484 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.113492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.113740 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.114453 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:46:57.114476 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:46:57.116781 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:46:57.120343 (+ 3562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:57.120354 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:57.120374 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:57.120863 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:57.121524 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:46:57.121527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:46:57.121530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:46:57.121814 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:46:57.122561 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:46:57.122567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:46:57.122997 (+ 430us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.123298 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.123769 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":295,"threads_started":1}
I20260809 22:46:57.126842 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.111754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.21.27.190:40159)
0809 22:46:57.112302 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.112308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.112719 (+ 411us) server_negotiation.cc:407] Connection header received
0809 22:46:57.112862 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.112868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.112956 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.113071 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.114617 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.115411 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.116850 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.117371 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.117515 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.120201 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:57.120206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:57.120209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:57.120242 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:57.123850 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.124723 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.124729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.124732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.124881 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.125287 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.125292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.125295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.125644 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:57.125844 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.126262 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.126521 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":290,"threads_started":1}
I20260809 22:46:57.126983 5414 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40159
I20260809 22:46:57.127372 5414 heartbeater.cc:464] Registering TS with master...
I20260809 22:46:57.128396 5414 heartbeater.cc:511] Master 127.21.27.190:40159 requested a full tablet report, sending...
I20260809 22:46:57.129961 4924 ts_manager.cc:194] Registered new tserver with Master: ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269)
I20260809 22:46:57.130789 4924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:47695
I20260809 22:46:57.142524 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:46:57.149705 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.143510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.21.27.190:40159)
0809 22:46:57.143709 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.143714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.144220 (+ 506us) server_negotiation.cc:407] Connection header received
0809 22:46:57.144434 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.144437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.144585 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.144732 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:57.145141 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.145849 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.147580 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.148016 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.148119 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.148639 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:57.149185 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:57.149306 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.149407 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.149487 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:46:57.150715 4924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:46:57.163110 4924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:46:57.186596 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.177680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.21.27.130:39679)
0809 22:46:57.178461 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.178471 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.178501 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:46:57.178609 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.178617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.178840 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.179789 (+ 949us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:57.180031 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.181631 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.184515 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.185436 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.185613 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.185804 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.186066 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.186235 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":491,"threads_started":1}
I20260809 22:46:57.186651 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.177320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39679 (local address 127.0.0.1:49682)
0809 22:46:57.177893 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.177912 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.178025 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.179081 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.179086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.179122 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:57.179548 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.179562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.181786 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.181793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.184341 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.184360 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.185678 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.185684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.185873 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.185897 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.186000 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.186211 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":33,"thread_start_us":302,"threads_started":1}
I20260809 22:46:57.186657 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.175841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.21.27.131:43269)
0809 22:46:57.176653 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.176663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.176700 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:46:57.177056 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.177064 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.177283 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.177486 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:57.179377 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.180806 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.183499 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.184974 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.185084 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.185328 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.186060 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.186277 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":462,"threads_started":1}
I20260809 22:46:57.187199 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.177966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:44364)
0809 22:46:57.178475 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.178493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.178618 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.179804 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.179808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.179832 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:57.180218 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.180233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.182273 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.182281 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.184395 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.184408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.186293 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.186299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.186503 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.186539 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.186609 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.186775 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":22,"thread_start_us":663,"threads_started":1}
I20260809 22:46:57.188551 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.178437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.21.27.129:32793)
0809 22:46:57.179221 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.179232 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.179262 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:46:57.179362 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.179371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.179584 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.180419 (+ 835us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:57.180683 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.182143 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.184560 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.186049 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.187752 (+ 1703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.187964 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.188042 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.188202 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":417,"threads_started":1}
I20260809 22:46:57.188583 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.175674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43269 (local address 127.0.0.1:56190)
0809 22:46:57.176560 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.176591 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.176798 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.178444 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.178451 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.178488 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:57.178933 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.178947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.180953 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.180956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.183318 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.183333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.185512 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.185520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.185776 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.185833 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.186932 (+ 1099us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.187072 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":27,"thread_start_us":958,"threads_started":1}
I20260809 22:46:57.191161 5200 tablet_service.cc:1467] Processing CreateTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 (DEFAULT_TABLE table=test-table [id=de78235637444a1587ed93557b233d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:57.191481 5343 tablet_service.cc:1467] Processing CreateTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 (DEFAULT_TABLE table=test-table [id=de78235637444a1587ed93557b233d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:57.192041 5057 tablet_service.cc:1467] Processing CreateTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 (DEFAULT_TABLE table=test-table [id=de78235637444a1587ed93557b233d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:46:57.192768 5343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de35bec19c6e4423bbfe9bc1eaddb425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:57.192893 5200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de35bec19c6e4423bbfe9bc1eaddb425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:57.193387 5057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de35bec19c6e4423bbfe9bc1eaddb425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:46:57.206684 5431 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Bootstrap starting.
I20260809 22:46:57.208829 5432 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Bootstrap starting.
I20260809 22:46:57.209345 5128 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
I20260809 22:46:57.210328 5433 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Bootstrap starting.
I20260809 22:46:57.210414 5431 tablet_bootstrap.cc:654] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:57.211782 5431 log.cc:824] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:57.214387 5433 tablet_bootstrap.cc:654] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:57.215605 5432 tablet_bootstrap.cc:654] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Neither blocks nor log segments found. Creating new log.
I20260809 22:46:57.215835 5433 log.cc:824] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:57.217118 5432 log.cc:824] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:57.219003 5433 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: No bootstrap required, opened a new log
I20260809 22:46:57.219372 5433 ts_tablet_manager.cc:1397] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 22:46:57.220150 5432 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: No bootstrap required, opened a new log
I20260809 22:46:57.220429 5432 ts_tablet_manager.cc:1397] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 22:46:57.220638 5431 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: No bootstrap required, opened a new log
I20260809 22:46:57.220920 5431 ts_tablet_manager.cc:1397] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260809 22:46:57.227629 5432 raft_consensus.cc:348] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.228029 5432 raft_consensus.cc:374] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:57.228129 5432 raft_consensus.cc:729] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ff2376ab0345d1af69705ac16d7e87, State: Initialized, Role: FOLLOWER
I20260809 22:46:57.228767 5432 consensus_queue.cc:260] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.230310 5433 raft_consensus.cc:348] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.230669 5432 ts_tablet_manager.cc:1428] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 22:46:57.230741 5433 raft_consensus.cc:374] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:57.230834 5433 raft_consensus.cc:729] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d0389561a54330ae02b65758312fc1, State: Initialized, Role: FOLLOWER
I20260809 22:46:57.231460 5433 consensus_queue.cc:260] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.233157 5270 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
I20260809 22:46:57.233341 5433 ts_tablet_manager.cc:1428] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 22:46:57.233249 5431 raft_consensus.cc:348] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.233690 5431 raft_consensus.cc:374] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:46:57.233830 5431 raft_consensus.cc:729] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0afcabd9dc4ac9ae37496b6224d127, State: Initialized, Role: FOLLOWER
I20260809 22:46:57.234395 5431 consensus_queue.cc:260] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.235749 5414 heartbeater.cc:503] Master 127.21.27.190:40159 was elected leader, sending a full tablet report...
I20260809 22:46:57.236320 5431 ts_tablet_manager.cc:1428] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260809 22:46:57.312742 5437 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:46:57.313133 5437 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.315519 5437 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
I20260809 22:46:57.323753 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.315998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:43597 (local address 127.21.27.131:43269)
0809 22:46:57.316284 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.316292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.316830 (+ 538us) server_negotiation.cc:407] Connection header received
0809 22:46:57.317129 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.317135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.317220 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.317296 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:57.318306 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.319332 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.321583 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.322688 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.322827 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.322881 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.323301 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.323464 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:46:57.323760 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.315939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43269 (local address 127.21.27.129:43597)
0809 22:46:57.316717 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.316738 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.316955 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.317552 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.317560 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.317600 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:57.318090 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.318104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.319495 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.319503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.321421 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.321434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.322909 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.322917 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.323099 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.323134 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.323278 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.323439 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":395,"threads_started":1}
I20260809 22:46:57.324230 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.317052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:36467 (local address 127.21.27.130:39679)
0809 22:46:57.317328 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.317337 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.317988 (+ 651us) server_negotiation.cc:407] Connection header received
0809 22:46:57.318209 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.318213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.318311 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.318410 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:46:57.318943 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.319812 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.322262 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.323275 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.323424 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.323492 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.323853 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.323999 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:46:57.324565 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.316783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39679 (local address 127.21.27.129:36467)
0809 22:46:57.317864 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.317879 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:46:57.318045 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:46:57.318432 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:46:57.318437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:46:57.318461 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:46:57.318780 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.318789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.319954 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.319960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.322056 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:46:57.322073 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.323491 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.323497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:46:57.323666 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.323696 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:46:57.323809 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 22:46:57.324111 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":740,"threads_started":1}
I20260809 22:46:57.325042 5364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de35bec19c6e4423bbfe9bc1eaddb425" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" is_pre_election: true
I20260809 22:46:57.325551 5364 raft_consensus.cc:2454] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 0.
I20260809 22:46:57.325506 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de35bec19c6e4423bbfe9bc1eaddb425" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11d0389561a54330ae02b65758312fc1" is_pre_election: true
I20260809 22:46:57.325956 5221 raft_consensus.cc:2454] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 0.
I20260809 22:46:57.326287 5002 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed0afcabd9dc4ac9ae37496b6224d127, f0ff2376ab0345d1af69705ac16d7e87; no voters:
I20260809 22:46:57.326732 5437 raft_consensus.cc:2781] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:46:57.326855 5437 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:46:57.326995 5437 raft_consensus.cc:3037] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:57.329257 5437 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.329844 5437 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 election: Requested vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
I20260809 22:46:57.330334 5364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de35bec19c6e4423bbfe9bc1eaddb425" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
I20260809 22:46:57.330410 5221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de35bec19c6e4423bbfe9bc1eaddb425" candidate_uuid: "f0ff2376ab0345d1af69705ac16d7e87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11d0389561a54330ae02b65758312fc1"
I20260809 22:46:57.330572 5364 raft_consensus.cc:3037] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:57.330662 5221 raft_consensus.cc:3037] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:46:57.333007 5221 raft_consensus.cc:2454] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 1.
I20260809 22:46:57.333069 5364 raft_consensus.cc:2454] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0ff2376ab0345d1af69705ac16d7e87 in term 1.
I20260809 22:46:57.333628 5002 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11d0389561a54330ae02b65758312fc1, f0ff2376ab0345d1af69705ac16d7e87; no voters:
I20260809 22:46:57.333945 5437 raft_consensus.cc:2781] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:46:57.334429 5437 raft_consensus.cc:686] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 LEADER]: Becoming Leader. State: Replica: f0ff2376ab0345d1af69705ac16d7e87, State: Running, Role: LEADER
I20260809 22:46:57.334928 5437 consensus_queue.cc:237] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:57.339968 4923 catalog_manager.cc:5495] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "f0ff2376ab0345d1af69705ac16d7e87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } health_report { overall_health: HEALTHY } } }
W20260809 22:46:57.353694 5415 tablet.cc:2369] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:46:57.373005 5271 tablet.cc:2369] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:46:57.389396 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.383550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.21.27.129:32793)
0809 22:46:57.383781 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.383791 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.383887 (+ 96us) server_negotiation.cc:407] Connection header received
0809 22:46:57.384167 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.384172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.384258 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.384419 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:46:57.384812 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.385543 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.387026 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.387504 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.387650 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.388244 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:46:57.388776 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:46:57.388937 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.389027 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.389117 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:46:57.398530 5364 raft_consensus.cc:1261] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Refusing update from remote peer f0ff2376ab0345d1af69705ac16d7e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:57.398643 5221 raft_consensus.cc:1261] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Refusing update from remote peer f0ff2376ab0345d1af69705ac16d7e87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:46:57.399760 5443 consensus_queue.cc:1035] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:57.400223 5437 consensus_queue.cc:1035] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:46:57.413636 5446 mvcc.cc:205] Tried to move back new op lower bound from 7316748768853266432 to 7316748768608018432. Current Snapshot: MvccSnapshot[applied={T|T < 7316748768853266432}]
I20260809 22:46:57.421411 5447 mvcc.cc:205] Tried to move back new op lower bound from 7316748768853266432 to 7316748768608018432. Current Snapshot: MvccSnapshot[applied={T|T < 7316748768853266432}]
I20260809 22:46:57.429528 4924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:57.429862 4924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:46:57.433887 4924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Sending DeleteTablet for 3 replicas of tablet de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:46:57.435233 5343 tablet_service.cc:1514] Processing DeleteTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:57 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:56190
I20260809 22:46:57.435500 5200 tablet_service.cc:1514] Processing DeleteTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:57 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:49682
I20260809 22:46:57.435536 5057 tablet_service.cc:1514] Processing DeleteTablet for tablet de35bec19c6e4423bbfe9bc1eaddb425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:46:57 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:44364
I20260809 22:46:57.436046 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4965
I20260809 22:46:57.437704 5455 tablet_replica.cc:330] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: stopping tablet replica
I20260809 22:46:57.438216 5455 raft_consensus.cc:2229] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:57.438835 5455 raft_consensus.cc:2258] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:57.442624 5455 ts_tablet_manager.cc:1905] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:57.450331 5455 ts_tablet_manager.cc:1918] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:57.450538 5455 log.cc:1196] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:46:57.451260 5455 ts_tablet_manager.cc:1939] T de35bec19c6e4423bbfe9bc1eaddb425 P ed0afcabd9dc4ac9ae37496b6224d127: Deleting consensus metadata
I20260809 22:46:57.453464 4908 catalog_manager.cc:4841] TS ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): tablet de35bec19c6e4423bbfe9bc1eaddb425 (table test-table [id=de78235637444a1587ed93557b233d96]) successfully deleted
I20260809 22:46:57.457275 5456 tablet_replica.cc:330] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: stopping tablet replica
I20260809 22:46:57.457783 5456 raft_consensus.cc:2229] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:46:57.458385 5456 raft_consensus.cc:2258] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:46:57.461483 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.437077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.21.27.129:32793)
0809 22:46:57.437307 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.437316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.437339 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:46:57.437512 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.437518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.437621 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.437778 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.440518 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.442223 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.444126 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.445136 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.445306 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.449081 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:57.449101 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:57.449128 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:57.449195 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:57.453950 (+ 4755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.455325 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.455335 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.455339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.455561 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.456105 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.456110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.456114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.456595 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:57.460506 (+ 3911us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.460689 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.460980 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:46:57.462145 5456 ts_tablet_manager.cc:1905] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:46:57.462252 5457 tablet_replica.cc:330] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: stopping tablet replica
I20260809 22:46:57.462598 5121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:57.470260 5456 ts_tablet_manager.cc:1918] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:46:57.470489 5456 log.cc:1196] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:46:57.471089 5456 ts_tablet_manager.cc:1939] T de35bec19c6e4423bbfe9bc1eaddb425 P 11d0389561a54330ae02b65758312fc1: Deleting consensus metadata
I20260809 22:46:57.473285 4908 catalog_manager.cc:4841] TS 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): tablet de35bec19c6e4423bbfe9bc1eaddb425 (table test-table [id=de78235637444a1587ed93557b233d96]) successfully deleted
I20260809 22:46:57.462733 5457 raft_consensus.cc:2229] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:46:57.562597 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4965
I20260809 22:46:57.562556 5457 raft_consensus.cc:2258] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 22:46:57.584682 4908 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.27.129:32793 (error 108)
I20260809 22:46:57.584841 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 5132
I20260809 22:46:57.586645 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.586229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49020)
0809 22:46:57.586403 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.586505 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 22:46:57.586870 4908 catalog_manager.cc:4584] TS f0ff2376ab0345d1af69705ac16d7e87 (127.21.27.129:32793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de35bec19c6e4423bbfe9bc1eaddb425: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
I20260809 22:46:57.600347 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.585706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.21.27.130:39679)
0809 22:46:57.585881 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.585888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.585996 (+ 108us) server_negotiation.cc:407] Connection header received
0809 22:46:57.586228 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.586234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.586313 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.586424 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.588063 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.588824 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.590259 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.590678 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.590822 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.593823 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:57.593839 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:57.593856 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:57.593902 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:57.596899 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.597739 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.597746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.597749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.597884 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.598312 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.598318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.598322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.598751 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:57.599063 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.599756 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.599989 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:46:57.601025 5265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:57.640650 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.640158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49022)
0809 22:46:57.640356 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.640490 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:46:57.641866 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.641447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49028)
0809 22:46:57.641658 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.641745 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 22:46:57.702628 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.702128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49042)
0809 22:46:57.702374 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.702472 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 22:46:57.703773 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.703412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49052)
0809 22:46:57.703564 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.703667 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 22:46:57.707456 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 5132
I20260809 22:46:57.726881 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 5273
I20260809 22:46:57.742554 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.727804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.21.27.131:43269)
0809 22:46:57.728006 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:46:57.728014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:46:57.728077 (+ 63us) server_negotiation.cc:407] Connection header received
0809 22:46:57.728367 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:46:57.728372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:46:57.728449 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:46:57.728561 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:46:57.730011 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.730754 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.732265 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:46:57.732680 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:46:57.732814 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:46:57.736671 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:46:57.736688 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:46:57.736705 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:46:57.736742 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:46:57.739820 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.740623 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.740631 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.740633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.740779 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:46:57.741125 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:46:57.741128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:46:57.741129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:46:57.741476 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:46:57.741713 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:46:57.741959 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 22:46:57.742195 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:46:57.743289 5407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:57.784669 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.784149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49060)
0809 22:46:57.784376 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.784469 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:46:57.785781 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.785391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49068)
0809 22:46:57.785595 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.785668 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 22:46:57.833846 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 5273
I20260809 22:46:57.851686 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 4889
I20260809 22:46:57.852973 4958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:46:57.970949 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:57.970032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:32793 (local address 127.0.0.1:49082)
0809 22:46:57.970427 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:57.970557 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:32793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260809 22:46:57.971303 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 4889
I20260809 22:46:57.997275 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--webserver_interface=127.21.27.190
--webserver_port=45981
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:46:58.337592 5465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:58.337878 5465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:58.337968 5465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:58.345484 5465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:46:58.345592 5465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:58.345638 5465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:58.345700 5465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:46:58.345742 5465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:46:58.355487 5465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=45981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:58.356727 5465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:58.357923 5465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:58.370553 5470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:58.370522 5471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:58.370815 5473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:58.371587 5465 server_base.cc:1034] running on GCE node
I20260809 22:46:58.372336 5465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:58.373749 5465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:58.375007 5465 hybrid_clock.cc:648] HybridClock initialized: now 1786315618374956 us; error 61 us; skew 500 ppm
I20260809 22:46:58.375407 5465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:58 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:58.378612 5465 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:46:58.380234 5465 webserver.cc:460] Webserver started at http://127.21.27.190:45981/ using document root <none> and password file <none>
I20260809 22:46:58.380935 5465 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:58.381090 5465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:58.386886 5465 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:46:58.389849 5480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:58.391341 5465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:46:58.391538 5465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "31043e5525c14155b07927952a0d74c3"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "f7e95af540fd28b8cdacd30fa490cd08"
server_key_iv: "556ac4f5ca7e9e212ce14d5f81d687b7"
server_key_version: "encryptionkey@0"
I20260809 22:46:58.392575 5465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:58.420926 5465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:58.425594 5465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:58.425917 5465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:58.446933 5465 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40159
I20260809 22:46:58.446952 5539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40159 every 8 connection(s)
I20260809 22:46:58.448624 5465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:46:58.455646 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5465
I20260809 22:46:58.456816 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:32793
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=44635
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:58.457202 5540 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:46:58.462872 5540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap starting.
I20260809 22:46:58.485886 5540 log.cc:824] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:58.505460 5540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:58.506093 5540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3: Bootstrap complete.
I20260809 22:46:58.515738 5540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:58.516719 5540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Initialized, Role: FOLLOWER
I20260809 22:46:58.517328 5540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:58.517575 5540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:46:58.517727 5540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:46:58.517915 5540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:46:58.522045 5540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:58.522576 5540 leader_election.cc:302] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31043e5525c14155b07927952a0d74c3; no voters:
I20260809 22:46:58.523314 5540 leader_election.cc:288] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:46:58.523465 5544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:46:58.524741 5544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [term 3 LEADER]: Becoming Leader. State: Replica: 31043e5525c14155b07927952a0d74c3, State: Running, Role: LEADER
I20260809 22:46:58.525367 5544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } }
I20260809 22:46:58.525630 5540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:46:58.527904 5546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31043e5525c14155b07927952a0d74c3. Latest consensus state: current_term: 3 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:58.528438 5546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:58.528757 5545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "31043e5525c14155b07927952a0d74c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31043e5525c14155b07927952a0d74c3" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40159 } } }
I20260809 22:46:58.529050 5545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31043e5525c14155b07927952a0d74c3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:46:58.531942 5553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:46:58.540741 5553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:46:58.542789 5553 catalog_manager.cc:1259] Loaded cluster ID: 4baf5afe14b74cef880aad255d05fc7a
I20260809 22:46:58.542994 5553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 22:46:58.545857 5553 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc6ea9bf9ad google::LogMessage::Fail() at ??:0
@ 0x7fc6ea9c288a google::LogMessage::SendToLog() at ??:0
@ 0x7fc6ea9bf4de google::LogMessage::Flush() at ??:0
@ 0x7fc6ea9c0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc7003f2512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc7003ef976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc700477610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc700477439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc6ed60b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc6ed60e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc6ed60e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc6ed5e9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc6f08c36db start_thread at ??:0
@ 0x7fc6e985961f clone at ??:0
W20260809 22:46:58.845343 5542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:58.845682 5542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:58.845860 5542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:58.853331 5542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:58.853498 5542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:58.853677 5542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:46:58.863822 5542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:32793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=44635
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:58.865191 5542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:58.866568 5542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:58.880028 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:58.880048 5589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:58.880097 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:58.881057 5542 server_base.cc:1034] running on GCE node
I20260809 22:46:58.881734 5542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:58.882846 5542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:58.884120 5542 hybrid_clock.cc:648] HybridClock initialized: now 1786315618884076 us; error 54 us; skew 500 ppm
I20260809 22:46:58.884466 5542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:58 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:58.887809 5542 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:46:58.889430 5542 webserver.cc:460] Webserver started at http://127.21.27.129:44635/ using document root <none> and password file <none>
I20260809 22:46:58.890159 5542 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:58.890319 5542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:58.895965 5542 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:46:58.898895 5597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:58.900283 5542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:46:58.900470 5542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "f0ff2376ab0345d1af69705ac16d7e87"
format_stamp: "Formatted at 2026-08-09 22:46:52 on dist-test-slave-chsk"
server_key: "59e0a7a7419932f4da63b5e4eeb53a20"
server_key_iv: "b1c5564a053aab1b6c80c7039fbe1df2"
server_key_version: "encryptionkey@0"
I20260809 22:46:58.901479 5542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:58.927243 5542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:58.931535 5542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:58.931839 5542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:58.932921 5542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:58.934839 5604 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 22:46:58.940614 5542 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 22:46:58.940768 5542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 22:46:58.940914 5542 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 22:46:58.944306 5542 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 22:46:58.944423 5542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260809 22:46:58.944816 5604 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Bootstrap starting.
I20260809 22:46:58.973179 5604 log.cc:824] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Log is configured to *not* fsync() on all Append() calls
I20260809 22:46:58.989230 5604 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:46:58.990037 5604 tablet_bootstrap.cc:492] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Bootstrap complete.
I20260809 22:46:58.990721 5542 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:32793
I20260809 22:46:58.990764 5721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:32793 every 8 connection(s)
I20260809 22:46:58.991667 5604 ts_tablet_manager.cc:1397] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Time spent bootstrapping tablet: real 0.047s user 0.021s sys 0.024s
I20260809 22:46:58.992146 5542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:46:58.997696 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:58.995040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:60841)
0809 22:46:58.996071 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:58.996441 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":398,"threads_started":1}
W20260809 22:46:58.998535 5722 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:40159 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:40159: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
I20260809 22:46:58.998804 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5542
I20260809 22:46:58.999648 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:58.999206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:34589)
0809 22:46:58.999408 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:58.999502 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:46:58.999662 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:39679
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=36551
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:59.000808 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.000357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:39747)
0809 22:46:59.000532 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.000637 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 22:46:59.001083 5722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:46:59.001082 5604 raft_consensus.cc:348] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:59.002170 5604 raft_consensus.cc:729] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0ff2376ab0345d1af69705ac16d7e87, State: Initialized, Role: FOLLOWER
I20260809 22:46:59.002785 5604 consensus_queue.cc:260] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:46:59.004487 5604 ts_tablet_manager.cc:1428] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260809 22:46:59.004675 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.004200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:39223)
0809 22:46:59.004379 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.004474 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:46:59.005710 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.005267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:57577)
0809 22:46:59.005445 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.005537 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 22:46:59.347318 5726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:59.347741 5726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:59.347963 5726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:59.355719 5726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:59.355855 5726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:59.356041 5726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:46:59.366506 5726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:39679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=36551
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:59.367930 5726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:59.369372 5726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:59.383229 5735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:59.383265 5732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:59.383302 5731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:59.384359 5726 server_base.cc:1034] running on GCE node
I20260809 22:46:59.385008 5726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:59.385996 5726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:59.387377 5726 hybrid_clock.cc:648] HybridClock initialized: now 1786315619387293 us; error 59 us; skew 500 ppm
I20260809 22:46:59.387710 5726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:59 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:59.391083 5726 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:46:59.392858 5726 webserver.cc:460] Webserver started at http://127.21.27.130:36551/ using document root <none> and password file <none>
I20260809 22:46:59.393599 5726 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:59.393746 5726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:59.399693 5726 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:46:59.402686 5743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.404233 5726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:46:59.404433 5726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "11d0389561a54330ae02b65758312fc1"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "5d3b41168a2f2ea316cdc02eee16f68f"
server_key_iv: "a05f33ca331dbcabf6df6df3c9a4fa8b"
server_key_version: "encryptionkey@0"
I20260809 22:46:59.405544 5726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:59.424245 5726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:59.428422 5726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:59.428722 5726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:59.429756 5726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:59.431107 5726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:59.431216 5726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.431375 5726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:59.431468 5726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.465867 5726 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:39679
I20260809 22:46:59.465963 5863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:39679 every 8 connection(s)
I20260809 22:46:59.467406 5726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:46:59.470283 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5726
I20260809 22:46:59.470963 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:43269
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:40159
--webserver_port=43763
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:46:59.472424 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.470387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:45819)
0809 22:46:59.471287 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.471556 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":353,"threads_started":1}
W20260809 22:46:59.473132 5864 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:40159 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:40159: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
I20260809 22:46:59.474013 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.473700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:35881)
0809 22:46:59.473842 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.473911 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 22:46:59.474822 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.474501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:39355)
0809 22:46:59.474636 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.474720 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:46:59.475025 5864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:46:59.810082 5867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:46:59.810400 5867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:46:59.810498 5867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:46:59.817548 5867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:46:59.817660 5867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:46:59.817773 5867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:46:59.827394 5867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:43269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=43763
--tserver_master_addrs=127.21.27.190:40159
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:46:59.828688 5867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:46:59.829967 5867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:46:59.842950 5873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:59.842905 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:46:59.842983 5875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:46:59.844141 5867 server_base.cc:1034] running on GCE node
I20260809 22:46:59.844781 5867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:46:59.845741 5867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:46:59.846972 5867 hybrid_clock.cc:648] HybridClock initialized: now 1786315619846925 us; error 42 us; skew 500 ppm
I20260809 22:46:59.847340 5867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:46:59 dist-test-slave-chsk krb5kdc[4292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315619, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:46:59.850538 5867 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:46:59.852003 5867 webserver.cc:460] Webserver started at http://127.21.27.131:43763/ using document root <none> and password file <none>
I20260809 22:46:59.852684 5867 fs_manager.cc:359] Metadata directory not provided
I20260809 22:46:59.852831 5867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:46:59.857900 5867 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:46:59.860617 5883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.861974 5867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:46:59.862197 5867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "ed0afcabd9dc4ac9ae37496b6224d127"
format_stamp: "Formatted at 2026-08-09 22:46:53 on dist-test-slave-chsk"
server_key: "67f19388df5ef35269f9bc4b83ff5344"
server_key_iv: "95d9201841eaf8ac8d295aa1db7c284f"
server_key_version: "encryptionkey@0"
I20260809 22:46:59.863252 5867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:46:59.887341 5867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:46:59.891510 5867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:46:59.891793 5867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:46:59.892818 5867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:46:59.894172 5867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:46:59.894309 5867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.894449 5867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:46:59.894541 5867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:46:59.929155 5867 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:43269
I20260809 22:46:59.929246 6002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:43269 every 8 connection(s)
I20260809 22:46:59.930696 5867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:46:59.935604 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.933682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:54173)
0809 22:46:59.934464 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.934699 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":329,"threads_started":1}
W20260809 22:46:59.936312 6003 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:40159 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:40159: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
I20260809 22:46:59.937142 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.936835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:39969)
0809 22:46:59.936969 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.937043 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:46:59.938027 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:46:59.937703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:38511)
0809 22:46:59.937830 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 22:46:59.937901 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:46:59.938280 6003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:46:59.940423 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 5867
I20260809 22:47:00.007853 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.006655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:45047)
0809 22:47:00.007456 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.007618 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":251,"threads_started":1}
I20260809 22:47:00.341040 6008 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:00.341336 6008 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:47:00.343290 6008 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
Aug 09 22:47:00 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
Aug 09 22:47:00 dist-test-slave-chsk krb5kdc[4292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:47:00.370297 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.344625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:48257 (local address 127.21.27.130:39679)
0809 22:47:00.345746 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:00.345766 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:00.346316 (+ 550us) server_negotiation.cc:407] Connection header received
0809 22:47:00.348409 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:00.348418 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:00.348893 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:00.349266 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:00.350877 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.352183 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.353961 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.354591 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.354823 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.362845 (+ 8022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:00.362853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:00.362864 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:00.362910 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:00.366352 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:00.367263 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:00.367270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:00.367277 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:00.367480 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:00.367918 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:00.367922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:00.367925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:00.368559 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:00.368698 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:00.369425 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 22:47:00.369857 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":701,"threads_started":1}
I20260809 22:47:00.370298 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.343859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:33897 (local address 127.21.27.131:43269)
0809 22:47:00.346087 (+ 2228us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:00.346098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:00.346133 (+ 35us) server_negotiation.cc:407] Connection header received
0809 22:47:00.348382 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:00.348397 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:00.348894 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:00.349266 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:00.350802 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.352227 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.353966 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.354584 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.354805 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.362617 (+ 7812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:00.362631 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:00.362655 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:00.362720 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:00.365697 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:00.366762 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:00.366771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:00.366778 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:00.366989 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:00.367577 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:00.367581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:00.367584 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:00.368302 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:00.368484 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:00.369424 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 22:47:00.369857 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2072,"thread_start_us":1901,"threads_started":1}
I20260809 22:47:00.370345 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.343644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:43269 (local address 127.21.27.129:33897)
0809 22:47:00.343947 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.343986 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:00.344597 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:00.349341 (+ 4744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:00.349349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:00.349935 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:00.350572 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.350596 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.352417 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.352421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.353807 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.353820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.354824 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.354831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.355128 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.359438 (+ 4310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:00.359470 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:00.362241 (+ 2771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:00.365866 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:00.365877 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:00.365892 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:00.366617 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:00.367162 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:00.367170 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:00.367173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:00.367416 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:00.368457 (+ 1041us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:00.368463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:00.368816 (+ 353us) client_negotiation.cc:769] Sending connection context
0809 22:47:00.369189 (+ 373us) client_negotiation.cc:240] Negotiation successful
0809 22:47:00.369825 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":290,"spinlock_wait_cycles":7424}
I20260809 22:47:00.370352 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.344071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:39679 (local address 127.21.27.129:48257)
0809 22:47:00.346196 (+ 2125us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.346211 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:00.346308 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:00.349851 (+ 3543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:00.349855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:00.350364 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:00.350665 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.350702 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.352392 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.352400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.353803 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:00.353818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:00.354874 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:00.354882 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:00.355122 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:00.359451 (+ 4329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:00.359495 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:00.362578 (+ 3083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:00.366562 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:00.366567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:00.366570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:00.367077 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:00.367626 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:00.367632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:00.367635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:00.367798 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:00.368703 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:00.368707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:00.368854 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 22:47:00.369189 (+ 335us) client_negotiation.cc:240] Negotiation successful
0809 22:47:00.369940 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2023,"mutex_wait_us":275,"spinlock_wait_cycles":28416,"thread_start_us":1885,"threads_started":1}
W20260809 22:47:00.374081 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
W20260809 22:47:00.374404 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:47:00.374516 5598 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0ff2376ab0345d1af69705ac16d7e87; no voters: 11d0389561a54330ae02b65758312fc1, ed0afcabd9dc4ac9ae37496b6224d127
I20260809 22:47:00.374949 6008 raft_consensus.cc:2726] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:47:00.476621 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.475638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:39553)
0809 22:47:00.476224 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.476372 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":285,"threads_started":1}
I20260809 22:47:00.939566 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:00.938779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:57879)
0809 22:47:00.939249 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:00.939357 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":202,"threads_started":1}
I20260809 22:47:01.009573 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.008794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:60097)
0809 22:47:01.009325 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.009411 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":295,"threads_started":1}
I20260809 22:47:01.478331 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.477616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:57887)
0809 22:47:01.478095 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.478178 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":259,"threads_started":1}
I20260809 22:47:01.941300 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:01.940496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:40451)
0809 22:47:01.941011 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:01.941116 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":270,"threads_started":1}
I20260809 22:47:02.011404 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.010484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:46859)
0809 22:47:02.011016 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.011131 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":246,"threads_started":1}
I20260809 22:47:02.163892 6019 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:02.164146 6019 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:47:02.165419 6019 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
W20260809 22:47:02.166010 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
W20260809 22:47:02.166290 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:47:02.166409 5598 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0ff2376ab0345d1af69705ac16d7e87; no voters: 11d0389561a54330ae02b65758312fc1, ed0afcabd9dc4ac9ae37496b6224d127
I20260809 22:47:02.166776 6019 raft_consensus.cc:2726] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:47:02.480005 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.479325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:52727)
0809 22:47:02.479765 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.479844 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260809 22:47:02.942914 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:02.942231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:38321)
0809 22:47:02.942676 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:02.942758 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":237,"threads_started":1}
I20260809 22:47:03.013017 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.012329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:58313)
0809 22:47:03.012764 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.012849 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":233,"threads_started":1}
I20260809 22:47:03.481851 6023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.481032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:54975)
0809 22:47:03.481507 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.481596 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":239,"threads_started":1}
I20260809 22:47:03.944723 6024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:03.943886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:50619)
0809 22:47:03.944390 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:03.944488 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":254,"threads_started":1}
I20260809 22:47:04.014958 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.014243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:45233)
0809 22:47:04.014680 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.014761 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260809 22:47:04.483630 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.482903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:56181)
0809 22:47:04.483377 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.483470 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":274,"threads_started":1}
I20260809 22:47:04.946410 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:04.945718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:45519)
0809 22:47:04.946175 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:04.946260 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":252,"threads_started":1}
I20260809 22:47:05.016552 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.015823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:52525)
0809 22:47:05.016298 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.016392 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":260,"threads_started":1}
I20260809 22:47:05.125265 6029 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:05.125478 6029 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:47:05.126057 6029 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
W20260809 22:47:05.127115 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
W20260809 22:47:05.127396 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:47:05.127527 5598 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0ff2376ab0345d1af69705ac16d7e87; no voters: 11d0389561a54330ae02b65758312fc1, ed0afcabd9dc4ac9ae37496b6224d127
I20260809 22:47:05.127816 6029 raft_consensus.cc:2726] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:47:05.485183 6031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.484469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:60099)
0809 22:47:05.484919 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.485001 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260809 22:47:05.948186 6032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:05.947453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:37483)
0809 22:47:05.947914 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:05.947995 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260809 22:47:06.018306 6033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.017527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:38125)
0809 22:47:06.017990 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.018083 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260809 22:47:06.487401 6034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.486111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:42675)
0809 22:47:06.487002 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.487118 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":265,"threads_started":1}
I20260809 22:47:06.950053 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:06.949178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:33395)
0809 22:47:06.949731 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:06.949813 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":247,"threads_started":1}
I20260809 22:47:07.020160 6036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.019416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:40453)
0809 22:47:07.019881 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.019985 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":233,"threads_started":1}
I20260809 22:47:07.489017 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.488322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:53119)
0809 22:47:07.488747 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.488825 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":224,"threads_started":1}
I20260809 22:47:07.951754 6039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:07.950966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:56877)
0809 22:47:07.951481 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:07.951575 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":303,"threads_started":1}
I20260809 22:47:08.022090 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:08.021203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:38053)
0809 22:47:08.021745 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:08.021874 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":319,"threads_started":1}
I20260809 22:47:08.279101 6041 raft_consensus.cc:482] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:08.279408 6041 raft_consensus.cc:504] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed0afcabd9dc4ac9ae37496b6224d127" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 43269 } } peers { permanent_uuid: "11d0389561a54330ae02b65758312fc1" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 39679 } } peers { permanent_uuid: "f0ff2376ab0345d1af69705ac16d7e87" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 32793 } }
I20260809 22:47:08.280030 6041 leader_election.cc:288] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269), 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679)
W20260809 22:47:08.280964 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed0afcabd9dc4ac9ae37496b6224d127 (127.21.27.131:43269): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
W20260809 22:47:08.281217 5598 leader_election.cc:341] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 11d0389561a54330ae02b65758312fc1 (127.21.27.130:39679): Not found: Tablet not found: de35bec19c6e4423bbfe9bc1eaddb425
I20260809 22:47:08.281322 5598 leader_election.cc:302] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0ff2376ab0345d1af69705ac16d7e87; no voters: 11d0389561a54330ae02b65758312fc1, ed0afcabd9dc4ac9ae37496b6224d127
I20260809 22:47:08.281636 6041 raft_consensus.cc:2726] T de35bec19c6e4423bbfe9bc1eaddb425 P f0ff2376ab0345d1af69705ac16d7e87 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:47:08.490729 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:08.489972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:41937)
0809 22:47:08.490486 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:08.490565 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":272,"threads_started":1}
I20260809 22:47:08.953413 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:08.952665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:33187)
0809 22:47:08.953163 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:08.953247 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":255,"threads_started":1}
I20260809 22:47:09.023906 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.023223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:37843)
0809 22:47:09.023658 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.023740 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260809 22:47:09.492388 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.491681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:49523)
0809 22:47:09.492118 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.492200 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":236,"threads_started":1}
I20260809 22:47:09.955276 6046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:09.954407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:47333)
0809 22:47:09.954907 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:09.954993 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":285,"threads_started":1}
I20260809 22:47:10.025571 6047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.024841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:53939)
0809 22:47:10.025309 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.025392 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260809 22:47:10.494048 6048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.493263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:45305)
0809 22:47:10.493780 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.493870 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":254,"threads_started":1}
I20260809 22:47:10.957190 6049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:10.956382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:43237)
0809 22:47:10.956927 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:10.957017 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":264,"threads_started":1}
I20260809 22:47:11.027529 6050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.026581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:51889)
0809 22:47:11.027101 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:11.027274 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":271,"threads_started":1}
I20260809 22:47:11.495950 6053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.495115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:38879)
0809 22:47:11.495683 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:11.495769 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":300,"threads_started":1}
I20260809 22:47:11.959092 6054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:11.958307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:36563)
0809 22:47:11.958818 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:11.958904 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":262,"threads_started":1}
I20260809 22:47:12.029176 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:12.028466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:44735)
0809 22:47:12.028923 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:12.029001 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260809 22:47:12.497639 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:12.496779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:54751)
0809 22:47:12.497361 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:12.497466 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":292,"threads_started":1}
I20260809 22:47:12.960824 6057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:12.960066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:54335)
0809 22:47:12.960538 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:12.960657 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":232,"threads_started":1}
I20260809 22:47:13.030840 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.030093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:55007)
0809 22:47:13.030560 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.030644 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":248,"threads_started":1}
I20260809 22:47:13.499411 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.498554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:35673)
0809 22:47:13.498999 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.499089 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":249,"threads_started":1}
I20260809 22:47:13.962658 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:13.961883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:45229)
0809 22:47:13.962359 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:13.962452 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":258,"threads_started":1}
I20260809 22:47:14.032925 6061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.032126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.129:37263)
0809 22:47:14.032637 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:14.032734 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":257,"threads_started":1}
I20260809 22:47:14.501477 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.500638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.130:54663)
0809 22:47:14.501186 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:14.501280 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":294,"threads_started":1}
I20260809 22:47:14.941671 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 5542
I20260809 22:47:14.957742 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.942869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.21.27.129:32793)
0809 22:47:14.943538 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:14.943549 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:14.943608 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:47:14.943688 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:14.943695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:14.944005 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:14.944200 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:14.946093 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.947123 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.948448 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:14.949049 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:14.949209 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:14.951971 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:14.951983 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:14.952009 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:14.952051 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:14.954739 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:14.955555 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:14.955564 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:14.955568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:14.955728 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:14.956131 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:14.956138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:14.956145 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:14.956479 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:14.956814 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:14.957133 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 22:47:14.957412 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":315,"threads_started":1}
I20260809 22:47:14.958973 5717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:14.964721 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:14.963848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40159 (local address 127.21.27.131:57955)
0809 22:47:14.964345 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:14.964477 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260809 22:47:15.051339 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 5542
I20260809 22:47:15.071501 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 5726
I20260809 22:47:15.085935 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:15.072294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.21.27.130:39679)
0809 22:47:15.072909 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:15.072914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:15.072930 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:47:15.072985 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:15.072989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:15.073057 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:15.073149 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:15.074574 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:15.075467 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:15.076676 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:15.077102 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:15.077197 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:15.080015 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:15.080020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:15.080023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:15.080062 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:15.082897 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:15.083912 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:15.083917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:15.083920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:15.084098 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:15.084450 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:15.084454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:15.084456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:15.084753 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:15.084974 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:15.085229 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 22:47:15.085583 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":387,"threads_started":1}
I20260809 22:47:15.087028 5859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:15.169945 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 5726
I20260809 22:47:15.190282 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 5867
I20260809 22:47:15.204250 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:15.191109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.21.27.131:43269)
0809 22:47:15.191749 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:15.191754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:15.191771 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:47:15.191826 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:15.191830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:15.191898 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:15.192002 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:15.193465 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:15.194353 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:15.195671 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:15.196078 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:15.196171 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:15.198806 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:15.198811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:15.198814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:15.198844 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:15.201549 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:15.202202 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:15.202206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:15.202208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:15.202340 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:15.202727 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:15.202730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:15.202732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:15.202979 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:15.203458 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:15.203746 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 22:47:15.203941 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":310,"threads_started":1}
I20260809 22:47:15.204908 5998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:15.288319 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 5867
2026-08-09T22:47:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25090 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:47:15 dist-test-slave-chsk krb5kdc[6073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:47:15 dist-test-slave-chsk krb5kdc[6073](info): set up 2 sockets
Aug 09 22:47:15 dist-test-slave-chsk krb5kdc[6073](info): commencing operation
krb5kdc: starting...
W20260809 22:47:16.472369 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:47:16 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:47:16.762872 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:17.105983 6087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:17.106280 6087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:17.106376 6087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:17.114428 6087 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:47:17.114531 6087 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:47:17.114578 6087 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:47:17.114632 6087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:47:17.114686 6087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:17.114756 6087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:17.114802 6087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:47:17.114850 6087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:47:17.125056 6087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:17.126314 6087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:17.127662 6087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:17.140277 6095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:17.140391 6094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:17.140367 6097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:17.140537 6087 server_base.cc:1034] running on GCE node
I20260809 22:47:17.141651 6087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:17.142973 6087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:17.144228 6087 hybrid_clock.cc:648] HybridClock initialized: now 1786315637144160 us; error 45 us; skew 500 ppm
I20260809 22:47:17.144572 6087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:17 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:17.151924 6087 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:47:17.153466 6087 webserver.cc:460] Webserver started at http://127.21.27.190:36521/ using document root <none> and password file <none>
I20260809 22:47:17.154237 6087 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:17.154377 6087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:17.154727 6087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:17.157896 6087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "c2acb266e049457eb7dfd54346eed899"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "7e4f46a9f16edf1cf05759b844ef048e"
server_key_iv: "925e30793a0601f5e860a84913316520"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.158715 6087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "c2acb266e049457eb7dfd54346eed899"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "7e4f46a9f16edf1cf05759b844ef048e"
server_key_iv: "925e30793a0601f5e860a84913316520"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.164261 6087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:47:17.167825 6104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:17.169173 6087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:47:17.169368 6087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "c2acb266e049457eb7dfd54346eed899"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "7e4f46a9f16edf1cf05759b844ef048e"
server_key_iv: "925e30793a0601f5e860a84913316520"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.169602 6087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:17.197911 6087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:17.204275 6087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:17.204597 6087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:17.224062 6087 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42547
I20260809 22:47:17.224086 6161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42547 every 8 connection(s)
I20260809 22:47:17.225455 6087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:47:17.229310 6162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:17.232790 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6087
I20260809 22:47:17.233024 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:47:17.233700 21614 external_mini_cluster.cc:1428] Setting key 54656c83db44f536da7d73926ec52ea4
I20260809 22:47:17.242478 6162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap starting.
I20260809 22:47:17.246419 6162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:17.247804 6162 log.cc:824] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:47:17 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:17.250801 6162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: No bootstrap required, opened a new log
I20260809 22:47:17.259318 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:17.235763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.21.27.190:42547)
0809 22:47:17.236560 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:17.236571 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:17.236611 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:47:17.238224 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:17.238245 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:17.238623 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:17.238935 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:17.240823 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.242546 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.244497 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.245158 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.245287 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:17.250431 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:17.250442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:17.250460 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:17.250504 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:17.254661 (+ 4157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:17.255441 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:17.255452 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:17.255465 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:17.255699 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:17.256077 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:17.256083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:17.256085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:17.256818 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:17.257128 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:17.257448 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 22:47:17.257920 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":387,"threads_started":1}
I20260809 22:47:17.261895 6162 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:17.262391 6162 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:17.262496 6162 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Initialized, Role: FOLLOWER
I20260809 22:47:17.263048 6162 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:17.263358 6162 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:47:17.263497 6162 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:47:17.263664 6162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:17.266208 6162 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:17.266700 6162 leader_election.cc:302] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2acb266e049457eb7dfd54346eed899; no voters:
I20260809 22:47:17.267351 6162 leader_election.cc:288] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:47:17.267498 6167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:17.268187 6167 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 LEADER]: Becoming Leader. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Running, Role: LEADER
I20260809 22:47:17.268739 6167 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:17.269387 6162 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:47:17.271664 6169 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2acb266e049457eb7dfd54346eed899. Latest consensus state: current_term: 1 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:47:17.271832 6168 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:47:17.272156 6169 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:17.272179 6168 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:17.275645 6172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:47:17.280534 6172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:47:17.288816 6172 catalog_manager.cc:1347] Generated new cluster ID: d1920207f5ae4a99b7bd48788a411354
I20260809 22:47:17.288937 6172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:47:17.306392 6172 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:47:17.307745 6172 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:47:17.313475 6172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Generated new TSK 0
I20260809 22:47:17.314509 6172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:47:17.382687 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:17.720800 6190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:17.721094 6190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:17.721261 6190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:17.728849 6190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:17.729008 6190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:17.729256 6190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:47:17.742511 6190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:17.743810 6190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:17.745016 6190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:17.757929 6196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:17.758044 6200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:17.757967 6197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:17.758852 6190 server_base.cc:1034] running on GCE node
I20260809 22:47:17.759537 6190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:17.760476 6190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:17.761683 6190 hybrid_clock.cc:648] HybridClock initialized: now 1786315637761618 us; error 50 us; skew 500 ppm
I20260809 22:47:17.762058 6190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:17 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:17.765285 6190 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:47:17.766687 6190 webserver.cc:460] Webserver started at http://127.21.27.129:35637/ using document root <none> and password file <none>
I20260809 22:47:17.767436 6190 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:17.767575 6190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:17.767895 6190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:17.771220 6190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "895cbc6146d44683b5390ba4ce1b2454"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "4751f5d32587a30d07fea512dca3e067"
server_key_iv: "c4c5cb2a2a5393d348dd97a31d5c7b39"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.772006 6190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "895cbc6146d44683b5390ba4ce1b2454"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "4751f5d32587a30d07fea512dca3e067"
server_key_iv: "c4c5cb2a2a5393d348dd97a31d5c7b39"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.777488 6190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:47:17.780887 6209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:17.782339 6190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:47:17.782503 6190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "895cbc6146d44683b5390ba4ce1b2454"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "4751f5d32587a30d07fea512dca3e067"
server_key_iv: "c4c5cb2a2a5393d348dd97a31d5c7b39"
server_key_version: "encryptionkey@0"
I20260809 22:47:17.782737 6190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:17.801137 6190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:17.805233 6190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:17.805537 6190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:17.806459 6190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:17.807874 6190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:17.807951 6190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:17.808121 6190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:17.808182 6190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:17.844336 6190 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:38655
I20260809 22:47:17.844369 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:38655 every 8 connection(s)
I20260809 22:47:17.845841 6190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:47:17.852809 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6190
I20260809 22:47:17.853042 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:47:17.853926 21614 external_mini_cluster.cc:1428] Setting key 6d7bdff90fad89272dd48f38f689ca4d
Aug 09 22:47:17 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:17.868094 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:17.848619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:59325 (local address 127.21.27.190:42547)
0809 22:47:17.849103 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:17.849108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:17.849969 (+ 861us) server_negotiation.cc:407] Connection header received
0809 22:47:17.851889 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:17.851895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:17.851991 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:17.852121 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:17.854255 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.855226 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.857473 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.857857 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.857967 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:17.862242 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:17.862247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:17.862250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:17.862283 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:17.864933 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:17.865685 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:17.865693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:17.865699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:17.865831 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:17.866280 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:17.866285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:17.866288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:17.866557 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:17.866719 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:17.867485 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 22:47:17.867724 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":278,"threads_started":1}
I20260809 22:47:17.869056 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:17.849091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:59325)
0809 22:47:17.849812 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:17.849852 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:17.851619 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:17.852373 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:17.852385 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:17.853220 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:17.854052 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:17.854081 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.855399 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.855403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:17.857306 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:17.857324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:17.858104 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:17.858111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:17.858447 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:17.859201 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:17.859240 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:17.861979 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:17.865100 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:17.865109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:17.865125 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:17.865560 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:17.865950 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:17.865954 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:17.865958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:17.866159 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:17.866693 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:17.866701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:17.867039 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 22:47:17.867393 (+ 354us) client_negotiation.cc:240] Negotiation successful
0809 22:47:17.867923 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":277,"threads_started":1}
I20260809 22:47:17.870662 6332 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:17.870968 6332 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:17.871770 6332 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:17.873976 6124 ts_manager.cc:194] Registered new tserver with Master: 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655)
I20260809 22:47:17.875604 6124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:59325
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:47:17.915001 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:18.241890 6338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:18.242174 6338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:18.242271 6338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:18.249496 6338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:18.249600 6338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:18.249720 6338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:47:18.259230 6338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:18.260392 6338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:18.261602 6338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:18.274834 6344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.274796 6345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.274797 6347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:18.275678 6338 server_base.cc:1034] running on GCE node
I20260809 22:47:18.276307 6338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:18.277253 6338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:18.278456 6338 hybrid_clock.cc:648] HybridClock initialized: now 1786315638278416 us; error 31 us; skew 500 ppm
I20260809 22:47:18.278781 6338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:18 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:18.282045 6338 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:47:18.283478 6338 webserver.cc:460] Webserver started at http://127.21.27.130:46479/ using document root <none> and password file <none>
I20260809 22:47:18.284133 6338 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:18.284265 6338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:18.284585 6338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:18.287779 6338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "1f3ba461976f48ce9058454cbb704f5c"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "fa6da2a7f9accd492fea28548f5cb11c"
server_key_iv: "5cd521920029a31a3e9476a5fc9702af"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.288532 6338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "1f3ba461976f48ce9058454cbb704f5c"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "fa6da2a7f9accd492fea28548f5cb11c"
server_key_iv: "5cd521920029a31a3e9476a5fc9702af"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.293488 6338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:47:18.296792 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.298017 6338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:47:18.298223 6338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "1f3ba461976f48ce9058454cbb704f5c"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "fa6da2a7f9accd492fea28548f5cb11c"
server_key_iv: "5cd521920029a31a3e9476a5fc9702af"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.298416 6338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:18.336097 6338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:18.340600 6338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:18.340929 6338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:18.341944 6338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:18.343376 6338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:18.343468 6338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.343619 6338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:18.343677 6338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.379411 6338 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:46065
I20260809 22:47:18.379444 6480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:46065 every 8 connection(s)
I20260809 22:47:18.380879 6338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:47:18.389372 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6338
I20260809 22:47:18.389590 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:47:18.390089 21614 external_mini_cluster.cc:1428] Setting key d047888dd386e76305c0027ea5769b36
Aug 09 22:47:18 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:18.401208 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.383500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:43897 (local address 127.21.27.190:42547)
0809 22:47:18.384013 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.384018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.384865 (+ 847us) server_negotiation.cc:407] Connection header received
0809 22:47:18.386787 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.386793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.386888 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.387010 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:18.388536 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.389279 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.391018 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.391722 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.391858 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:18.395353 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:18.395359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:18.395362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:18.395395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:18.398031 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.398757 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.398762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.398766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.398905 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.399419 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.399424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.399429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.399710 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:18.399869 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:18.400586 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 22:47:18.400823 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":297,"threads_started":1}
I20260809 22:47:18.402359 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.384008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:43897)
0809 22:47:18.384698 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:18.384742 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:18.386516 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:18.387278 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:18.387285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:18.387806 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:18.388354 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.388371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.389452 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.389456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:18.390848 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.390864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.391961 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.391971 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:18.392220 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:18.392914 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:18.392937 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:18.395070 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:18.398180 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:18.398189 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:18.398204 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:18.398635 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:18.399019 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:18.399024 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:18.399027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:18.399303 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:18.399840 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:18.399848 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:18.400224 (+ 376us) client_negotiation.cc:769] Sending connection context
0809 22:47:18.400483 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 22:47:18.401012 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":274,"threads_started":1}
I20260809 22:47:18.404109 6481 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:18.404451 6481 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:18.405412 6481 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:18.407177 6124 ts_manager.cc:194] Registered new tserver with Master: 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
I20260809 22:47:18.408494 6124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:43897
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:47:18.455866 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:18.784909 6487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:18.785218 6487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:18.785378 6487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:18.792770 6487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:18.792912 6487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:18.793099 6487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:47:18.802891 6487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:18.804132 6487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:18.805392 6487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:18.818358 6497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.818414 6494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:18.818320 6495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:18.819242 6487 server_base.cc:1034] running on GCE node
I20260809 22:47:18.819788 6487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:18.820638 6487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:18.821831 6487 hybrid_clock.cc:648] HybridClock initialized: now 1786315638821797 us; error 27 us; skew 500 ppm
I20260809 22:47:18.822206 6487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:18 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:18.825372 6487 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:47:18.826668 6487 webserver.cc:460] Webserver started at http://127.21.27.131:46297/ using document root <none> and password file <none>
I20260809 22:47:18.827358 6487 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:18.827466 6487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:18.827710 6487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:47:18.830443 6487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "eb55f31412594c92ba98ba8451e44afa"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "81b501589992dda8470cd0da33362890"
server_key_iv: "c2029543059f0a21a0c698c32e8f3622"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.831252 6487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "eb55f31412594c92ba98ba8451e44afa"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "81b501589992dda8470cd0da33362890"
server_key_iv: "c2029543059f0a21a0c698c32e8f3622"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.836542 6487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:47:18.839911 6504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.841240 6487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:47:18.841420 6487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "eb55f31412594c92ba98ba8451e44afa"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "81b501589992dda8470cd0da33362890"
server_key_iv: "c2029543059f0a21a0c698c32e8f3622"
server_key_version: "encryptionkey@0"
I20260809 22:47:18.841620 6487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:18.872304 6487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:18.877136 6487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:18.877449 6487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:18.878365 6332 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:18.878507 6487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:18.880095 6487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:18.880219 6487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.880354 6487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:18.880431 6487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:18.916376 6487 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:35735
I20260809 22:47:18.916426 6627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:35735 every 8 connection(s)
I20260809 22:47:18.917708 6487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:47:18.923125 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6487
I20260809 22:47:18.923396 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:47:18.923892 21614 external_mini_cluster.cc:1428] Setting key ab9f2b72b3b8f7826d26faf0191c02ba
Aug 09 22:47:18 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315638, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:18.939410 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.920382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:36335 (local address 127.21.27.190:42547)
0809 22:47:18.920898 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.920902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.921853 (+ 951us) server_negotiation.cc:407] Connection header received
0809 22:47:18.923893 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.923899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.924037 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.924181 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:18.926071 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.927010 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.928666 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.929075 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.929173 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:18.932915 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:18.932921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:18.932924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:18.932970 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:18.935857 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.936592 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.936596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.936598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.936730 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.937112 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.937115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.937117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.937389 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:18.937524 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:18.938740 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0809 22:47:18.938935 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":290,"threads_started":1}
I20260809 22:47:18.940505 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.920958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:36335)
0809 22:47:18.921670 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:18.921716 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:18.923614 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:18.924464 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:18.924476 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:18.925122 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:18.925875 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.925903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.927224 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.927231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:18.928527 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.928539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.929271 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.929276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:18.929470 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:18.930183 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:18.930206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:18.932679 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:18.935988 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:18.935995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:18.936007 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:18.936451 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:18.936827 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:18.936830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:18.936832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:18.937007 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:18.937793 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:18.937801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:18.938258 (+ 457us) client_negotiation.cc:769] Sending connection context
0809 22:47:18.938576 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 22:47:18.939116 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":290,"threads_started":1}
I20260809 22:47:18.941972 6628 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:18.942263 6628 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:18.942953 6628 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:18.944425 6124 ts_manager.cc:194] Registered new tserver with Master: eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735)
I20260809 22:47:18.945235 6124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:36335
I20260809 22:47:18.952481 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:47:18.970414 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.957355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.21.27.190:42547)
0809 22:47:18.957604 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.957612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.957943 (+ 331us) server_negotiation.cc:407] Connection header received
0809 22:47:18.958077 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.958081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.958151 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.958235 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:18.959803 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.960575 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.961964 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.962394 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.962509 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:18.965142 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:18.965147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:18.965151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:18.965183 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:18.967755 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.968461 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.968465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.968467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.968588 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:18.968909 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:18.968912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:18.968914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:18.969286 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:18.969462 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:18.969876 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 22:47:18.970043 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:47:18.974654 6124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:47:18.981601 6124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:47:19.005395 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.995027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.21.27.131:35735)
0809 22:47:18.995888 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.995899 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.995935 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:47:18.996042 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.996052 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.996334 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.996536 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:18.997640 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.999293 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.002830 (+ 3537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.004127 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.004293 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.004551 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.004865 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.005036 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":495,"threads_started":1}
I20260809 22:47:19.005406 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.994669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35735 (local address 127.0.0.1:55508)
0809 22:47:18.995542 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:18.995601 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:18.995804 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:18.996902 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:18.996909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:18.996942 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:18.997369 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.997384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:18.999466 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:18.999472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.002587 (+ 3115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.002603 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.004342 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.004349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.004569 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.004637 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.004793 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.005004 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":383,"threads_started":1}
I20260809 22:47:19.006434 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.997153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46065 (local address 127.0.0.1:52316)
0809 22:47:18.998051 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:18.998073 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:18.998269 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:18.998999 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:18.999003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:18.999028 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:18.999449 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.999470 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.000912 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.000918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.003375 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.003391 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.005521 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.005527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.005708 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.005738 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.005825 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.005959 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":5,"thread_start_us":637,"threads_started":1}
I20260809 22:47:19.006928 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.996831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.21.27.130:46065)
0809 22:47:18.997596 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.997603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.998165 (+ 562us) server_negotiation.cc:407] Connection header received
0809 22:47:18.998480 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.998487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.998719 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.998878 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:18.999659 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.000750 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.003596 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.005007 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.005154 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.005356 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.006385 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.006576 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":499,"threads_started":1}
I20260809 22:47:19.006970 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.995953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:55278)
0809 22:47:18.996560 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:18.996584 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:18.996728 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:18.997989 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:18.997994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:18.998021 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:18.998489 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:18.998502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.000758 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.000763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.003310 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.003326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.006119 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.006126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.006329 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.006362 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.006460 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.006591 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":70,"thread_start_us":281,"threads_started":1}
I20260809 22:47:19.007752 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:18.996238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.21.27.129:38655)
0809 22:47:18.997359 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:18.997371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:18.997400 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:47:18.997498 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:18.997505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:18.997753 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:18.998678 (+ 925us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:18.998938 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.000593 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.003474 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.005831 (+ 2357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.006025 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.006265 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.007207 (+ 942us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.007413 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":936,"thread_start_us":784,"threads_started":1}
I20260809 22:47:19.010574 6558 tablet_service.cc:1467] Processing CreateTablet for tablet ec99b01094504a208b5fd832566fd540 (DEFAULT_TABLE table=test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:19.011590 6409 tablet_service.cc:1467] Processing CreateTablet for tablet ec99b01094504a208b5fd832566fd540 (DEFAULT_TABLE table=test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:19.012331 6558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec99b01094504a208b5fd832566fd540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:19.012975 6409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec99b01094504a208b5fd832566fd540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:19.013401 6261 tablet_service.cc:1467] Processing CreateTablet for tablet ec99b01094504a208b5fd832566fd540 (DEFAULT_TABLE table=test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:19.014621 6261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec99b01094504a208b5fd832566fd540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:19.025971 6648 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Bootstrap starting.
I20260809 22:47:19.028146 6652 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Bootstrap starting.
I20260809 22:47:19.028992 6651 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Bootstrap starting.
I20260809 22:47:19.031767 6652 tablet_bootstrap.cc:654] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:19.031838 6648 tablet_bootstrap.cc:654] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:19.032496 6651 tablet_bootstrap.cc:654] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:19.032931 6652 log.cc:824] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:19.033178 6648 log.cc:824] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:19.033702 6651 log.cc:824] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:19.036059 6648 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: No bootstrap required, opened a new log
I20260809 22:47:19.036206 6652 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: No bootstrap required, opened a new log
I20260809 22:47:19.036392 6648 ts_tablet_manager.cc:1397] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 22:47:19.036489 6652 ts_tablet_manager.cc:1397] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:47:19.036791 6651 tablet_bootstrap.cc:492] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: No bootstrap required, opened a new log
I20260809 22:47:19.037081 6651 ts_tablet_manager.cc:1397] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 22:47:19.045267 6648 raft_consensus.cc:348] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.045763 6648 raft_consensus.cc:374] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:19.045892 6648 raft_consensus.cc:729] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895cbc6146d44683b5390ba4ce1b2454, State: Initialized, Role: FOLLOWER
I20260809 22:47:19.046600 6648 consensus_queue.cc:260] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.047375 6652 raft_consensus.cc:348] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.047896 6652 raft_consensus.cc:374] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:19.048004 6652 raft_consensus.cc:729] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3ba461976f48ce9058454cbb704f5c, State: Initialized, Role: FOLLOWER
I20260809 22:47:19.047886 6651 raft_consensus.cc:348] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.048313 6651 raft_consensus.cc:374] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:19.048424 6651 raft_consensus.cc:729] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb55f31412594c92ba98ba8451e44afa, State: Initialized, Role: FOLLOWER
I20260809 22:47:19.048745 6652 consensus_queue.cc:260] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.048975 6651 consensus_queue.cc:260] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.049922 6481 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:19.050933 6652 ts_tablet_manager.cc:1428] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 22:47:19.052569 6651 ts_tablet_manager.cc:1428] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260809 22:47:19.053050 6628 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:19.053551 6648 ts_tablet_manager.cc:1428] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Time spent starting tablet: real 0.017s user 0.011s sys 0.002s
W20260809 22:47:19.098464 6333 tablet.cc:2369] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:47:19.132980 6482 tablet.cc:2369] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:47:19.169497 6629 tablet.cc:2369] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:47:19.367127 6658 raft_consensus.cc:482] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:19.367537 6658 raft_consensus.cc:504] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.369273 6658 leader_election.cc:288] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
I20260809 22:47:19.375273 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.369739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:47243 (local address 127.21.27.129:38655)
0809 22:47:19.369994 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.370001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.370023 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:47:19.370137 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.370144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.370235 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.370359 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:19.371013 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.371817 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.373526 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.374408 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.374627 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.374689 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.374924 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.375017 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:47:19.375350 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.369645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.21.27.131:47243)
0809 22:47:19.369843 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:19.369869 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:19.369993 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:19.370570 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:19.370576 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:19.370602 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:19.370862 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.370870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.371953 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.371960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.373361 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.373373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.374577 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.374584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.374769 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.374810 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.374909 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.375045 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:47:19.376885 6659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.370067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46065 (local address 127.21.27.131:59787)
0809 22:47:19.370806 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:19.370820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:19.370959 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:19.371326 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:19.371331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:19.371353 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:19.371670 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.371678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.372693 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.372699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.374329 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:19.374341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.375933 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.375939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:19.376119 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.376148 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:47:19.376239 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:47:19.376475 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":410,"threads_started":1}
I20260809 22:47:19.376936 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec99b01094504a208b5fd832566fd540" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895cbc6146d44683b5390ba4ce1b2454" is_pre_election: true
I20260809 22:47:19.377590 6284 raft_consensus.cc:2454] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 0.
I20260809 22:47:19.378475 6505 leader_election.cc:302] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454, eb55f31412594c92ba98ba8451e44afa; no voters:
I20260809 22:47:19.379007 6658 raft_consensus.cc:2781] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:47:19.379197 6658 raft_consensus.cc:482] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:47:19.379335 6658 raft_consensus.cc:3037] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:19.382817 6658 raft_consensus.cc:504] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.383311 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.370271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:59787 (local address 127.21.27.130:46065)
0809 22:47:19.370488 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.370496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.370913 (+ 417us) server_negotiation.cc:407] Connection header received
0809 22:47:19.371077 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.371083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.371205 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.371313 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:19.371824 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.372527 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.374507 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.375740 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.375917 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.375985 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.376357 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.376504 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:47:19.383654 6658 leader_election.cc:288] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 election: Requested vote from peers 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
I20260809 22:47:19.384331 6432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec99b01094504a208b5fd832566fd540" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3ba461976f48ce9058454cbb704f5c" is_pre_election: true
I20260809 22:47:19.384654 6284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec99b01094504a208b5fd832566fd540" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895cbc6146d44683b5390ba4ce1b2454"
I20260809 22:47:19.384887 6284 raft_consensus.cc:3037] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:19.384938 6432 raft_consensus.cc:2454] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 0.
I20260809 22:47:19.385776 6432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec99b01094504a208b5fd832566fd540" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3ba461976f48ce9058454cbb704f5c"
I20260809 22:47:19.385963 6432 raft_consensus.cc:3037] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:19.388589 6284 raft_consensus.cc:2454] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 1.
I20260809 22:47:19.389060 6505 leader_election.cc:302] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454, eb55f31412594c92ba98ba8451e44afa; no voters:
I20260809 22:47:19.389353 6432 raft_consensus.cc:2454] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 1.
I20260809 22:47:19.389407 6658 raft_consensus.cc:2781] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:19.390087 6658 raft_consensus.cc:686] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 1 LEADER]: Becoming Leader. State: Replica: eb55f31412594c92ba98ba8451e44afa, State: Running, Role: LEADER
I20260809 22:47:19.390633 6658 consensus_queue.cc:237] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } }
I20260809 22:47:19.395803 6122 catalog_manager.cc:5495] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa reported cstate change: term changed from 0 to 1, leader changed from <none> to eb55f31412594c92ba98ba8451e44afa (127.21.27.131). New cstate: current_term: 1 leader_uuid: "eb55f31412594c92ba98ba8451e44afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } health_report { overall_health: UNKNOWN } } }
I20260809 22:47:19.468941 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.463718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.21.27.131:35735)
0809 22:47:19.463904 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.463908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.464019 (+ 111us) server_negotiation.cc:407] Connection header received
0809 22:47:19.464242 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.464246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.464327 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.464457 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:47:19.464805 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.465538 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.466970 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.467391 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.467482 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.467953 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:47:19.468459 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:47:19.468568 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.468665 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.468733 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:47:19.477236 6432 raft_consensus.cc:1261] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Refusing update from remote peer eb55f31412594c92ba98ba8451e44afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:19.477236 6284 raft_consensus.cc:1261] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Refusing update from remote peer eb55f31412594c92ba98ba8451e44afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:19.478547 6660 consensus_queue.cc:1035] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:19.478943 6658 consensus_queue.cc:1035] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:19.505755 6124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:19.506103 6124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:19.509986 6124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Sending DeleteTablet for 3 replicas of tablet ec99b01094504a208b5fd832566fd540
I20260809 22:47:19.511107 6558 tablet_service.cc:1514] Processing DeleteTablet for tablet ec99b01094504a208b5fd832566fd540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:19 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:55508
I20260809 22:47:19.511365 6409 tablet_service.cc:1514] Processing DeleteTablet for tablet ec99b01094504a208b5fd832566fd540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:19 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:52316
I20260809 22:47:19.511519 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6190
I20260809 22:47:19.512351 6261 tablet_service.cc:1514] Processing DeleteTablet for tablet ec99b01094504a208b5fd832566fd540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:19 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:55278
I20260809 22:47:19.523851 6673 tablet_replica.cc:330] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: stopping tablet replica
I20260809 22:47:19.524544 6673 raft_consensus.cc:2229] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:47:19.525368 6673 raft_consensus.cc:2258] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:19.527846 6673 ts_tablet_manager.cc:1905] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:19.535982 6673 ts_tablet_manager.cc:1918] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:19.536295 6673 log.cc:1196] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/ec99b01094504a208b5fd832566fd540
I20260809 22:47:19.536999 6673 ts_tablet_manager.cc:1939] T ec99b01094504a208b5fd832566fd540 P eb55f31412594c92ba98ba8451e44afa: Deleting consensus metadata
I20260809 22:47:19.538987 6676 tablet_replica.cc:330] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: stopping tablet replica
I20260809 22:47:19.539088 6105 catalog_manager.cc:4841] TS eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): tablet ec99b01094504a208b5fd832566fd540 (table test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]) successfully deleted
Aug 09 22:47:19 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:47:19.539677 6676 raft_consensus.cc:2229] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:19.540215 6674 tablet_replica.cc:330] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: stopping tablet replica
I20260809 22:47:19.540246 6676 raft_consensus.cc:2258] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:19.540750 6674 raft_consensus.cc:2229] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:19.541317 6674 raft_consensus.cc:2258] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:19.542387 6676 ts_tablet_manager.cc:1905] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:19.543448 6674 ts_tablet_manager.cc:1905] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:19.550457 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.512840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.21.27.129:38655)
0809 22:47:19.513077 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.513086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.525314 (+ 12228us) server_negotiation.cc:407] Connection header received
0809 22:47:19.525856 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.525864 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.525974 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.526086 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:19.528705 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.530160 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.532852 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.533732 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.533853 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.541945 (+ 8092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:19.541958 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:19.541975 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:19.542054 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:19.546297 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.547641 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.547650 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.547654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.547846 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.548349 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.548355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.548364 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.548845 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:19.549166 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.549792 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.550068 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:47:19.550074 6676 ts_tablet_manager.cc:1918] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:19.550899 6676 log.cc:1196] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/ec99b01094504a208b5fd832566fd540
I20260809 22:47:19.550946 6674 ts_tablet_manager.cc:1918] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:19.551249 6674 log.cc:1196] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/ec99b01094504a208b5fd832566fd540
I20260809 22:47:19.551561 6676 ts_tablet_manager.cc:1939] T ec99b01094504a208b5fd832566fd540 P 895cbc6146d44683b5390ba4ce1b2454: Deleting consensus metadata
I20260809 22:47:19.551846 6327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:19.551961 6674 ts_tablet_manager.cc:1939] T ec99b01094504a208b5fd832566fd540 P 1f3ba461976f48ce9058454cbb704f5c: Deleting consensus metadata
I20260809 22:47:19.553995 6105 catalog_manager.cc:4841] TS 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): tablet ec99b01094504a208b5fd832566fd540 (table test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]) successfully deleted
I20260809 22:47:19.647210 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6190
I20260809 22:47:19.647222 6105 catalog_manager.cc:4841] TS 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655): tablet ec99b01094504a208b5fd832566fd540 (table test-table [id=9782abe85dcd4c9ab1b442c492ec9e17]) successfully deleted
I20260809 22:47:19.666926 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6338
Aug 09 22:47:19 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:47:19.685652 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.667830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52326 (local address 127.21.27.130:46065)
0809 22:47:19.668009 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.668016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.668072 (+ 56us) server_negotiation.cc:407] Connection header received
0809 22:47:19.668274 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.668281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.668365 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.668474 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:19.669892 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.671085 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.672854 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.673538 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.673677 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.678688 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:19.678711 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:19.678732 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:19.678781 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:19.682627 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.683402 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.683412 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.683416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.683583 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.683974 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.683979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.683983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.684492 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:19.684796 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.684999 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.685272 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:47:19.686488 6475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:19.779979 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6338
I20260809 22:47:19.799921 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6487
Aug 09 22:47:19 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:47:19.819281 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:19.800769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.21.27.131:35735)
0809 22:47:19.800983 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:19.800989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:19.801011 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:47:19.801178 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:19.801186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:19.801284 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:19.801411 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:19.802796 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.804059 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.805406 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:19.806080 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:19.806231 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:19.811311 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:19.811327 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:19.811347 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:19.811406 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:19.815684 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.816649 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.816656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.816658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.816830 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:19.817262 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:19.817266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:19.817268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:19.817817 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:19.818190 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:19.818734 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 22:47:19.818899 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 22:47:19.820048 6623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:19.944208 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6487
I20260809 22:47:19.968252 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6087
I20260809 22:47:19.969560 6156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:20.084017 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6087
I20260809 22:47:40.109520 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--webserver_interface=127.21.27.190
--webserver_port=36521
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:47:40.447440 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:40.447783 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:40.447947 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:40.455184 6681 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:47:40.455333 6681 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:47:40.455395 6681 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:47:40.455452 6681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:47:40.455507 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:40.455569 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:40.455614 6681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:47:40.455662 6681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:47:40.465399 6681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:40.466602 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:40.467935 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:40.493324 6691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:40.493366 6688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:40.493324 6687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:40.496644 6681 server_base.cc:1034] running on GCE node
I20260809 22:47:40.497594 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:40.499351 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:40.500587 6681 hybrid_clock.cc:648] HybridClock initialized: now 1786315660500508 us; error 46 us; skew 500 ppm
I20260809 22:47:40.501000 6681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:40 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:40.508419 6681 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:47:40.510408 6681 webserver.cc:460] Webserver started at http://127.21.27.190:36521/ using document root <none> and password file <none>
I20260809 22:47:40.511340 6681 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:40.511489 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:40.519275 6681 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 22:47:40.522966 6699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:40.524394 6681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:47:40.524588 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "c2acb266e049457eb7dfd54346eed899"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "7e4f46a9f16edf1cf05759b844ef048e"
server_key_iv: "925e30793a0601f5e860a84913316520"
server_key_version: "encryptionkey@0"
I20260809 22:47:40.525719 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:40.538822 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:40.543558 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:40.543877 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:40.563898 6681 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42547
I20260809 22:47:40.563983 6753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42547 every 8 connection(s)
I20260809 22:47:40.565512 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:47:40.570259 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6681
I20260809 22:47:40.571112 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:38655
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=35637
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:47:40.573190 6754 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:47:40.576565 6754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap starting.
I20260809 22:47:40.596726 6754 log.cc:824] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:40.609635 6754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:47:40.610296 6754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap complete.
I20260809 22:47:40.620146 6754 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:40.621090 6754 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Initialized, Role: FOLLOWER
I20260809 22:47:40.621740 6754 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:40.622028 6754 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:47:40.622186 6754 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:47:40.622381 6754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:47:40.626605 6754 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:40.627310 6754 leader_election.cc:302] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2acb266e049457eb7dfd54346eed899; no voters:
I20260809 22:47:40.627983 6754 leader_election.cc:288] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:47:40.628139 6758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:47:40.629668 6758 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 LEADER]: Becoming Leader. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Running, Role: LEADER
I20260809 22:47:40.630410 6758 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:47:40.630646 6754 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:47:40.633059 6760 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2acb266e049457eb7dfd54346eed899. Latest consensus state: current_term: 2 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:47:40.633446 6760 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:40.633258 6759 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:47:40.634119 6759 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:47:40.634807 6766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:47:40.650470 6766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:47:40.652098 6766 catalog_manager.cc:1259] Loaded cluster ID: d1920207f5ae4a99b7bd48788a411354
I20260809 22:47:40.652271 6766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:47:40.656786 6766 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:47:40.658393 6766 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Loaded TSK: 0
I20260809 22:47:40.702203 6766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Generated new TSK 1
I20260809 22:47:40.702865 6766 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:47:40.972973 6756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:40.973307 6756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:40.973480 6756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:40.981168 6756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:40.981326 6756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:40.981500 6756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:47:40.991503 6756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:38655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=35637
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:40.992868 6756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:40.994295 6756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:41.012017 6782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.012226 6783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.013046 6785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:41.013725 6756 server_base.cc:1034] running on GCE node
I20260809 22:47:41.014434 6756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:41.015548 6756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:41.016778 6756 hybrid_clock.cc:648] HybridClock initialized: now 1786315661016713 us; error 60 us; skew 500 ppm
I20260809 22:47:41.017131 6756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:41 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:41.020577 6756 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:47:41.022202 6756 webserver.cc:460] Webserver started at http://127.21.27.129:35637/ using document root <none> and password file <none>
I20260809 22:47:41.022925 6756 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:41.023089 6756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:41.028959 6756 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:47:41.032279 6795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.033761 6756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:47:41.033931 6756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "895cbc6146d44683b5390ba4ce1b2454"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "4751f5d32587a30d07fea512dca3e067"
server_key_iv: "c4c5cb2a2a5393d348dd97a31d5c7b39"
server_key_version: "encryptionkey@0"
I20260809 22:47:41.034929 6756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:41.051654 6756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:41.056246 6756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:41.056600 6756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:41.057780 6756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:41.059264 6756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:41.059393 6756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.059515 6756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:41.059607 6756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.097658 6756 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:38655
I20260809 22:47:41.097754 6920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:38655 every 8 connection(s)
I20260809 22:47:41.099426 6756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:47:41.102335 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6756
I20260809 22:47:41.103264 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:46065
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=46479
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:47:41 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:41.129784 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:41.102635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:37899 (local address 127.21.27.190:42547)
0809 22:47:41.103414 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:41.103432 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:41.103984 (+ 552us) server_negotiation.cc:407] Connection header received
0809 22:47:41.107405 (+ 3421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:41.107419 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:41.107783 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:41.108094 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:41.110316 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.111700 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.114225 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.115432 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.115640 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:41.120849 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:41.120858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:41.120874 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:41.120915 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:41.124433 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:41.125632 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:41.125641 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:41.125647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:41.125827 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:41.126437 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:41.126441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:41.126443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:41.126976 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:41.127253 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:41.128238 (+ 985us) server_negotiation.cc:299] Negotiation successful
0809 22:47:41.128559 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":282,"threads_started":1}
I20260809 22:47:41.129992 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:41.102782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:37899)
0809 22:47:41.103775 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:41.103827 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:41.106669 (+ 2842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:41.108312 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:41.108325 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:41.109213 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:41.110038 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:41.110069 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.111890 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.111898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:41.114040 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:41.114059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.115703 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.115711 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:41.116003 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:41.116952 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:41.116988 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:41.120492 (+ 3504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:41.124659 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:41.124673 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:41.124693 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:41.125446 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:41.125970 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:41.125977 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:41.125984 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:41.126287 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:41.127179 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:41.127191 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:41.127687 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 22:47:41.128014 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 22:47:41.128644 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":423,"threads_started":1}
I20260809 22:47:41.133402 6921 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:41.133723 6921 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:41.134429 6921 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:41.136962 6717 ts_manager.cc:194] Registered new tserver with Master: 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655)
I20260809 22:47:41.139168 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:37899
W20260809 22:47:41.444490 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:41.444823 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:41.444949 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:41.452870 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:41.453014 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:41.453152 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:47:41.463641 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:46065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=46479
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:41.464979 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:41.466279 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:41.480300 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.480361 6934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.480269 6932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:41.481252 6926 server_base.cc:1034] running on GCE node
I20260809 22:47:41.481845 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:41.482892 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:41.484097 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786315661484041 us; error 65 us; skew 500 ppm
I20260809 22:47:41.484436 6926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:41 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:41.488070 6926 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:47:41.489857 6926 webserver.cc:460] Webserver started at http://127.21.27.130:46479/ using document root <none> and password file <none>
I20260809 22:47:41.490592 6926 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:41.490723 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:41.496433 6926 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 22:47:41.499265 6942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.500671 6926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:47:41.500864 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "1f3ba461976f48ce9058454cbb704f5c"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "fa6da2a7f9accd492fea28548f5cb11c"
server_key_iv: "5cd521920029a31a3e9476a5fc9702af"
server_key_version: "encryptionkey@0"
I20260809 22:47:41.501874 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:41.528636 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:41.533277 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:41.533567 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:41.534615 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:41.536212 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:41.536311 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.536448 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:41.536535 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.574925 6926 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:46065
I20260809 22:47:41.575057 7066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:46065 every 8 connection(s)
I20260809 22:47:41.576371 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:47:41.578277 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 6926
I20260809 22:47:41.578967 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:35735
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=46297
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:47:41 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:41.601110 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:41.579379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:33075 (local address 127.21.27.190:42547)
0809 22:47:41.579596 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:41.579604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:41.580935 (+ 1331us) server_negotiation.cc:407] Connection header received
0809 22:47:41.583863 (+ 2928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:41.583868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:41.583959 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:41.584061 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:41.586160 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.586856 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.589336 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.589766 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.589890 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:41.593917 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:41.593926 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:41.593934 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:41.593983 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:41.597656 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:41.598411 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:41.598421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:41.598428 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:41.598590 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:41.599026 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:41.599034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:41.599041 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:41.599499 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:41.599656 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:41.600349 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 22:47:41.600621 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:47:41.601740 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:41.579836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:33075)
0809 22:47:41.580755 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:41.580808 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:41.583547 (+ 2739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:41.584318 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:41.584331 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:41.585155 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:41.585968 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:41.585995 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.587030 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.587037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:41.589159 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:41.589177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:41.590015 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:41.590025 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:41.590352 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:41.591189 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:41.591223 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:41.593651 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:41.597813 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:41.597820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:41.597833 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:41.598268 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:41.598703 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:41.598706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:41.598708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:41.598889 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:41.599648 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:41.599654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:41.599967 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 22:47:41.600196 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 22:47:41.600591 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":354,"threads_started":1}
I20260809 22:47:41.603252 7067 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:41.603592 7067 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:41.604494 7067 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:41.606335 6717 ts_manager.cc:194] Registered new tserver with Master: 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
I20260809 22:47:41.607357 6717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:33075
W20260809 22:47:41.922549 7070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:47:41.922832 7070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:47:41.922993 7070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:47:41.930258 7070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:47:41.930368 7070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:47:41.930572 7070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:47:41.940634 7070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:35735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=46297
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:47:41.941918 7070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:47:41.943238 7070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:47:41.956233 7076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.956208 7075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:47:41.956179 7078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:47:41.957064 7070 server_base.cc:1034] running on GCE node
I20260809 22:47:41.957700 7070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:47:41.958669 7070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:47:41.959892 7070 hybrid_clock.cc:648] HybridClock initialized: now 1786315661959846 us; error 47 us; skew 500 ppm
I20260809 22:47:41.960211 7070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:47:41 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:47:41.963464 7070 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:47:41.964838 7070 webserver.cc:460] Webserver started at http://127.21.27.131:46297/ using document root <none> and password file <none>
I20260809 22:47:41.965507 7070 fs_manager.cc:359] Metadata directory not provided
I20260809 22:47:41.965669 7070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:47:41.971189 7070 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 22:47:41.973869 7085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:41.975287 7070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:47:41.975461 7070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "eb55f31412594c92ba98ba8451e44afa"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "81b501589992dda8470cd0da33362890"
server_key_iv: "c2029543059f0a21a0c698c32e8f3622"
server_key_version: "encryptionkey@0"
I20260809 22:47:41.976434 7070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:47:41.992944 7070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:47:41.997397 7070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:47:41.997699 7070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:47:41.998764 7070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:47:42.000156 7070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:47:42.000278 7070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:42.000406 7070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:47:42.000499 7070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:47:42.034685 7070 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:35735
I20260809 22:47:42.034783 7207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:35735 every 8 connection(s)
I20260809 22:47:42.036266 7070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:47:42.043254 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7070
Aug 09 22:47:42 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:47:42.057766 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.039046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:56231 (local address 127.21.27.190:42547)
0809 22:47:42.039275 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.039280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.040496 (+ 1216us) server_negotiation.cc:407] Connection header received
0809 22:47:42.042289 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.042294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.042369 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.042462 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.044314 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.045284 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.047101 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.047817 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.047958 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.051659 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.051666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.051671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.051712 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.054362 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.055216 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.055223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.055227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.055395 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.055869 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.055874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.055877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.056251 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.056414 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.057120 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.057371 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:47:42.058897 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.039520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:56231)
0809 22:47:42.040299 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.040360 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.042019 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.042684 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.042692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.043356 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:47:42.044052 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.044082 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.045463 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.045471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.046941 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.046955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.048074 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.048084 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.048323 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.049150 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:47:42.049173 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:47:42.051439 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:47:42.054534 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:42.054544 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:42.054562 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:42.055071 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:42.055521 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:47:42.055526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:47:42.055530 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:47:42.055738 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:47:42.056389 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:47:42.056395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:47:42.056728 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.056955 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.057437 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":344,"threads_started":1}
I20260809 22:47:42.060441 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.044285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.21.27.190:42547)
0809 22:47:42.044976 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.044985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.045903 (+ 918us) server_negotiation.cc:407] Connection header received
0809 22:47:42.046102 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.046108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.046213 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.046325 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.048271 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.049245 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.050711 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.051274 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.051405 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.054433 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.054439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.054442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.054472 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.057466 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.058302 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.058306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.058308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.058455 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.058878 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.058881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.058883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.059225 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.059489 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.059828 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.060055 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":341,"threads_started":1}
I20260809 22:47:42.060551 7208 heartbeater.cc:347] Connected to a master server at 127.21.27.190:42547
I20260809 22:47:42.060858 7208 heartbeater.cc:464] Registering TS with master...
I20260809 22:47:42.061604 7208 heartbeater.cc:511] Master 127.21.27.190:42547 requested a full tablet report, sending...
I20260809 22:47:42.063304 6715 ts_manager.cc:194] Registered new tserver with Master: eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735)
I20260809 22:47:42.064193 6715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:56231
I20260809 22:47:42.076049 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:47:42.082929 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.076991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.21.27.190:42547)
0809 22:47:42.077270 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.077279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.077692 (+ 413us) server_negotiation.cc:407] Connection header received
0809 22:47:42.077884 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.077888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.078071 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.078201 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:47:42.078627 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.079476 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.080955 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.081484 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.081588 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.082064 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:47:42.082336 (+ 272us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 22:47:42.082646 (+ 310us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49810: token expired
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 22:47:42.083071 7211 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49810: token expired
W20260809 22:47:42.083627 6634 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.27.190:42547)
I20260809 22:47:42.097277 7211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.084233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.21.27.190:42547)
0809 22:47:42.084408 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.084415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.084435 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:47:42.084488 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.084491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.084544 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.084657 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.086031 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.086746 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.088323 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.088727 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.088870 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.091838 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.091847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.091850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.091882 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.094356 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.095080 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.095088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.095091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.095294 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.095711 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.095716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.095719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.096043 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.096308 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.096651 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.096900 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:47:42.101608 6715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:47:42.109745 6715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:47:42.134285 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.122999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.21.27.131:35735)
0809 22:47:42.123727 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.123737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.124495 (+ 758us) server_negotiation.cc:407] Connection header received
0809 22:47:42.124795 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.124803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.125054 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.125238 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:42.126574 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.128326 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.131329 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.132813 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.132963 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.133222 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.133755 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.133930 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":684,"threads_started":1}
I20260809 22:47:42.134532 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.122246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35735 (local address 127.0.0.1:54142)
0809 22:47:42.124325 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.124371 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.124580 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.125773 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.125782 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.125821 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:42.126346 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.126359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.128519 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.128525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.131099 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.131111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.133016 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.133023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.133416 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.133499 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.133666 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.133916 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1886,"mutex_wait_us":97,"thread_start_us":1762,"threads_started":1}
I20260809 22:47:42.136041 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.124912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46065 (local address 127.0.0.1:46436)
0809 22:47:42.125971 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.125991 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.126113 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.127020 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.127026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.127051 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:42.127446 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.127459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.129259 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.129264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.133152 (+ 3888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.133165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.135340 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.135346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.135525 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.135557 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.135652 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.135796 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":768,"threads_started":1}
I20260809 22:47:42.137107 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.125149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.21.27.130:46065)
0809 22:47:42.125815 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.125822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.126241 (+ 419us) server_negotiation.cc:407] Connection header received
0809 22:47:42.126341 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.126348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.126582 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.126764 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:42.127638 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.129088 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.133343 (+ 4255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.134854 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.134998 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.135192 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.136604 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.136770 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":332,"threads_started":1}
I20260809 22:47:42.139405 7137 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee834188b07420cb7057dc0c455ed26 (DEFAULT_TABLE table=test-table [id=4179eb802a1746f59e06c580dd88257b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:42.139848 6996 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee834188b07420cb7057dc0c455ed26 (DEFAULT_TABLE table=test-table [id=4179eb802a1746f59e06c580dd88257b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:42.141227 6996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee834188b07420cb7057dc0c455ed26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:42.141427 7137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee834188b07420cb7057dc0c455ed26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:42.142223 6921 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:42.151940 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.126081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50792)
0809 22:47:42.130180 (+ 4099us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.130197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.130300 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.144034 (+ 13734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.144042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.144073 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:42.144407 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.144418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.146415 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.146421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.148757 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.148778 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.151017 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.151024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.151237 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.151276 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.151378 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.151534 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3960,"thread_start_us":3802,"threads_started":1}
I20260809 22:47:42.152765 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.126436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.21.27.129:38655)
0809 22:47:42.143421 (+ 16985us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.143430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.143460 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:47:42.143552 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.143559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.143803 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.144017 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:42.144602 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.146215 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.148959 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.150453 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.150599 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.150861 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.152258 (+ 1397us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.152426 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16790,"thread_start_us":17754,"threads_started":1}
I20260809 22:47:42.155941 7225 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Bootstrap starting.
I20260809 22:47:42.156719 6846 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee834188b07420cb7057dc0c455ed26 (DEFAULT_TABLE table=test-table [id=4179eb802a1746f59e06c580dd88257b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:47:42.157873 6846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee834188b07420cb7057dc0c455ed26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:47:42.159916 7226 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Bootstrap starting.
I20260809 22:47:42.160404 7225 tablet_bootstrap.cc:654] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:42.161770 7225 log.cc:824] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:42.163439 7226 tablet_bootstrap.cc:654] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:42.164420 7225 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: No bootstrap required, opened a new log
I20260809 22:47:42.164629 7226 log.cc:824] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:42.164695 7225 ts_tablet_manager.cc:1397] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:47:42.167692 7226 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: No bootstrap required, opened a new log
I20260809 22:47:42.167928 7226 ts_tablet_manager.cc:1397] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 22:47:42.173075 7225 raft_consensus.cc:348] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.173450 7225 raft_consensus.cc:374] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:42.173554 7225 raft_consensus.cc:729] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3ba461976f48ce9058454cbb704f5c, State: Initialized, Role: FOLLOWER
I20260809 22:47:42.174176 7225 consensus_queue.cc:260] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.175299 7067 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:42.175845 7225 ts_tablet_manager.cc:1428] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260809 22:47:42.178725 7226 raft_consensus.cc:348] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.179250 7226 raft_consensus.cc:374] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:42.179376 7226 raft_consensus.cc:729] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb55f31412594c92ba98ba8451e44afa, State: Initialized, Role: FOLLOWER
I20260809 22:47:42.180176 7226 consensus_queue.cc:260] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.181573 7230 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Bootstrap starting.
I20260809 22:47:42.185093 7230 tablet_bootstrap.cc:654] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Neither blocks nor log segments found. Creating new log.
I20260809 22:47:42.185896 7208 heartbeater.cc:503] Master 127.21.27.190:42547 was elected leader, sending a full tablet report...
I20260809 22:47:42.186302 7230 log.cc:824] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Log is configured to *not* fsync() on all Append() calls
I20260809 22:47:42.186510 7226 ts_tablet_manager.cc:1428] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260809 22:47:42.188956 7230 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: No bootstrap required, opened a new log
I20260809 22:47:42.189229 7230 ts_tablet_manager.cc:1397] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 22:47:42.196280 7230 raft_consensus.cc:348] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.196672 7230 raft_consensus.cc:374] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:47:42.196774 7230 raft_consensus.cc:729] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895cbc6146d44683b5390ba4ce1b2454, State: Initialized, Role: FOLLOWER
I20260809 22:47:42.197358 7230 consensus_queue.cc:260] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.198966 7230 ts_tablet_manager.cc:1428] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
W20260809 22:47:42.287945 7209 tablet.cc:2369] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:47:42.328593 7068 tablet.cc:2369] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:47:42.350930 7231 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:47:42.351326 7231 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
W20260809 22:47:42.351902 6922 tablet.cc:2369] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:47:42.353147 7231 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065), 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655)
I20260809 22:47:42.359678 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.353526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46065 (local address 127.21.27.131:42709)
0809 22:47:42.353794 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.353817 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.353985 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.354487 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.354492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.354525 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:42.354703 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.354733 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.355981 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.355985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.357835 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.357850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.359104 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.359108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.359287 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.359312 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.359387 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.359478 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 22:47:42.359850 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.353622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:42709 (local address 127.21.27.130:46065)
0809 22:47:42.353828 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.353836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.353926 (+ 90us) server_negotiation.cc:407] Connection header received
0809 22:47:42.354171 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.354179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.354284 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.354376 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:42.354858 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.355858 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.357987 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.358962 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.359113 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.359220 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.359438 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.359568 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:47:42.360668 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.354134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:56071 (local address 127.21.27.129:38655)
0809 22:47:42.354354 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.354362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.354761 (+ 399us) server_negotiation.cc:407] Connection header received
0809 22:47:42.354923 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.354927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.354984 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.355079 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:47:42.355607 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.356435 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.358502 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.359672 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.359824 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.359909 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.360397 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.360490 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:47:42.360869 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.353975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.21.27.131:56071)
0809 22:47:42.354656 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.354671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:47:42.354774 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:47:42.355096 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:47:42.355101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:47:42.355125 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:47:42.355442 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.355451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.356569 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.356574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.358350 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:47:42.358361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.360046 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.360052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:47:42.360219 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.360242 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:47:42.360358 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 22:47:42.360498 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":1,"thread_start_us":405,"threads_started":1}
I20260809 22:47:42.360898 7017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee834188b07420cb7057dc0c455ed26" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3ba461976f48ce9058454cbb704f5c" is_pre_election: true
I20260809 22:47:42.361429 7017 raft_consensus.cc:2454] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 0.
I20260809 22:47:42.361853 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee834188b07420cb7057dc0c455ed26" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895cbc6146d44683b5390ba4ce1b2454" is_pre_election: true
I20260809 22:47:42.362113 7086 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3ba461976f48ce9058454cbb704f5c, eb55f31412594c92ba98ba8451e44afa; no voters:
I20260809 22:47:42.362413 6869 raft_consensus.cc:2454] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 0.
I20260809 22:47:42.362542 7231 raft_consensus.cc:2781] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:47:42.362718 7231 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:47:42.362849 7231 raft_consensus.cc:3037] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:42.365090 7231 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.365679 7231 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 election: Requested vote from peers 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065), 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655)
I20260809 22:47:42.366132 7017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee834188b07420cb7057dc0c455ed26" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3ba461976f48ce9058454cbb704f5c"
I20260809 22:47:42.366192 6869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee834188b07420cb7057dc0c455ed26" candidate_uuid: "eb55f31412594c92ba98ba8451e44afa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895cbc6146d44683b5390ba4ce1b2454"
I20260809 22:47:42.366324 7017 raft_consensus.cc:3037] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:42.366389 6869 raft_consensus.cc:3037] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:47:42.369006 7017 raft_consensus.cc:2454] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 1.
I20260809 22:47:42.369002 6869 raft_consensus.cc:2454] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb55f31412594c92ba98ba8451e44afa in term 1.
I20260809 22:47:42.369418 7086 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454, eb55f31412594c92ba98ba8451e44afa; no voters:
I20260809 22:47:42.369710 7231 raft_consensus.cc:2781] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:47:42.370153 7231 raft_consensus.cc:686] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 1 LEADER]: Becoming Leader. State: Replica: eb55f31412594c92ba98ba8451e44afa, State: Running, Role: LEADER
I20260809 22:47:42.370712 7231 consensus_queue.cc:237] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:47:42.375666 6715 catalog_manager.cc:5495] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa reported cstate change: term changed from 0 to 1, leader changed from <none> to eb55f31412594c92ba98ba8451e44afa (127.21.27.131). New cstate: current_term: 1 leader_uuid: "eb55f31412594c92ba98ba8451e44afa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } health_report { overall_health: UNKNOWN } } }
I20260809 22:47:42.436578 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.431603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.21.27.131:35735)
0809 22:47:42.431839 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.431843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.431860 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:47:42.431998 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.432004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.432093 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.432229 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:47:42.432569 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.433265 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.434570 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.434948 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.435037 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.435555 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:47:42.436030 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:47:42.436148 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.436215 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.436281 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 22:47:42.444525 6869 raft_consensus.cc:1261] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Refusing update from remote peer eb55f31412594c92ba98ba8451e44afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:42.444597 7017 raft_consensus.cc:1261] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Refusing update from remote peer eb55f31412594c92ba98ba8451e44afa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:47:42.445478 7231 consensus_queue.cc:1035] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:42.445849 7235 consensus_queue.cc:1035] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:47:42.472716 6715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:42.472980 6715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:47:42.476483 6715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Sending DeleteTablet for 3 replicas of tablet 0ee834188b07420cb7057dc0c455ed26
I20260809 22:47:42.477532 7137 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee834188b07420cb7057dc0c455ed26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:42 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:54142
I20260809 22:47:42.477789 6846 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee834188b07420cb7057dc0c455ed26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:42 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:50792
I20260809 22:47:42.477789 6996 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee834188b07420cb7057dc0c455ed26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:47:42 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:46436
I20260809 22:47:42.478685 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6756
I20260809 22:47:42.490051 7250 tablet_replica.cc:330] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: stopping tablet replica
I20260809 22:47:42.490511 7250 raft_consensus.cc:2229] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:42.491026 7250 raft_consensus.cc:2258] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:42.491561 7248 tablet_replica.cc:330] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: stopping tablet replica
I20260809 22:47:42.491966 7249 tablet_replica.cc:330] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: stopping tablet replica
I20260809 22:47:42.492024 7248 raft_consensus.cc:2229] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:47:42.492398 7249 raft_consensus.cc:2229] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:47:42.492754 7248 raft_consensus.cc:2258] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:42.492933 7249 raft_consensus.cc:2258] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:47:42.494110 7250 ts_tablet_manager.cc:1905] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:42.494829 7248 ts_tablet_manager.cc:1905] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:42.494829 7249 ts_tablet_manager.cc:1905] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:47:42.500896 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.479791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.21.27.129:38655)
0809 22:47:42.480085 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.480095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.480898 (+ 803us) server_negotiation.cc:407] Connection header received
0809 22:47:42.480979 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.480986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.481103 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.481213 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.483896 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.485441 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.488767 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.489741 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.489916 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.494945 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.494963 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.494979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.495023 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.497823 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.498733 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.498740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.498742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.498871 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.499339 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.499350 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.499353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.499687 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.499999 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.500285 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.500499 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 22:47:42.501519 6914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:42.501735 7249 ts_tablet_manager.cc:1918] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:42.501736 7248 ts_tablet_manager.cc:1918] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:47:42.501907 7248 log.cc:1196] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/0ee834188b07420cb7057dc0c455ed26
I20260809 22:47:42.501989 7249 log.cc:1196] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/0ee834188b07420cb7057dc0c455ed26
I20260809 22:47:42.502691 7249 ts_tablet_manager.cc:1939] T 0ee834188b07420cb7057dc0c455ed26 P 1f3ba461976f48ce9058454cbb704f5c: Deleting consensus metadata
I20260809 22:47:42.505352 6700 catalog_manager.cc:4841] TS 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): tablet 0ee834188b07420cb7057dc0c455ed26 (table test-table [id=4179eb802a1746f59e06c580dd88257b]) successfully deleted
I20260809 22:47:42.505854 7248 ts_tablet_manager.cc:1939] T 0ee834188b07420cb7057dc0c455ed26 P eb55f31412594c92ba98ba8451e44afa: Deleting consensus metadata
I20260809 22:47:42.507627 6700 catalog_manager.cc:4841] TS eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): tablet 0ee834188b07420cb7057dc0c455ed26 (table test-table [id=4179eb802a1746f59e06c580dd88257b]) successfully deleted
I20260809 22:47:42.601964 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6756
W20260809 22:47:42.622727 6700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.27.129:38655 (error 108)
I20260809 22:47:42.623606 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6926
I20260809 22:47:42.624771 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.624182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50822)
0809 22:47:42.624457 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.624588 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260809 22:47:42.625023 6700 catalog_manager.cc:4584] TS 895cbc6146d44683b5390ba4ce1b2454 (127.21.27.129:38655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ee834188b07420cb7057dc0c455ed26: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
I20260809 22:47:42.640373 7222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.624403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.21.27.130:46065)
0809 22:47:42.624583 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.624589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.624640 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:47:42.624867 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.624873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.624968 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.625080 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.626547 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.627918 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.629193 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.629883 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.630029 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.633062 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.633079 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.633099 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.633153 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.637085 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.637861 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.637872 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.637876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.638028 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.638638 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.638644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.638647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.639191 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.639505 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.639727 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.639967 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:47:42.641086 7060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:42.678956 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.678354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50834)
0809 22:47:42.678663 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.678779 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 22:47:42.680222 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.679775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50848)
0809 22:47:42.679955 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.680041 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:47:42.735719 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6926
I20260809 22:47:42.741178 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.740555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50850)
0809 22:47:42.740827 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.740944 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 22:47:42.742419 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.741979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50856)
0809 22:47:42.742174 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.742268 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 22:47:42.756554 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 7070
I20260809 22:47:42.772773 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.757371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.21.27.131:35735)
0809 22:47:42.757561 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:47:42.757570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:47:42.757596 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:47:42.757774 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:47:42.757798 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:47:42.757882 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:47:42.757999 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:47:42.759414 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.760657 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.761957 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:47:42.762704 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:47:42.762855 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:47:42.765484 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:47:42.765499 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:47:42.765520 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:47:42.765567 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:47:42.769650 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.770453 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.770467 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.770470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.770670 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:47:42.771060 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:47:42.771066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:47:42.771071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:47:42.771538 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:47:42.771902 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 22:47:42.772091 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 22:47:42.772338 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:47:42.773588 7201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:42.823534 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.822901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50862)
0809 22:47:42.823187 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.823304 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 22:47:42.824678 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:42.824247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50878)
0809 22:47:42.824456 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:42.824550 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 22:47:42.896636 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 7070
I20260809 22:47:42.917883 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 6681
I20260809 22:47:42.919035 6749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:47:43.035722 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 6681
I20260809 22:47:43.035854 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:47:43.035017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:38655 (local address 127.0.0.1:50882)
0809 22:47:43.035324 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 22:47:43.035469 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:38655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260809 22:48:03.062191 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--webserver_interface=127.21.27.190
--webserver_port=36521
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:03.405777 7256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:03.406152 7256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:03.406327 7256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:03.413901 7256 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 22:48:03.414161 7256 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 22:48:03.414275 7256 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 22:48:03.414391 7256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:03.414500 7256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:03.414640 7256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:03.414731 7256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:03.414813 7256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:03.425369 7256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=36521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:03.426775 7256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:03.428159 7256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:03.441045 7262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:03.440932 7263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:03.441313 7265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:03.441924 7256 server_base.cc:1034] running on GCE node
I20260809 22:48:03.442791 7256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:03.444399 7256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:03.445677 7256 hybrid_clock.cc:648] HybridClock initialized: now 1786315683445600 us; error 51 us; skew 500 ppm
I20260809 22:48:03.446058 7256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:03 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315683, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:03.449416 7256 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:48:03.451002 7256 webserver.cc:460] Webserver started at http://127.21.27.190:36521/ using document root <none> and password file <none>
I20260809 22:48:03.451776 7256 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:03.451913 7256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:03.457597 7256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:48:03.460556 7273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:03.461917 7256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:03.462141 7256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "c2acb266e049457eb7dfd54346eed899"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "7e4f46a9f16edf1cf05759b844ef048e"
server_key_iv: "925e30793a0601f5e860a84913316520"
server_key_version: "encryptionkey@0"
I20260809 22:48:03.463114 7256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:03.486720 7256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:03.490989 7256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:03.491374 7256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:03.510905 7256 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42547
I20260809 22:48:03.510957 7328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42547 every 8 connection(s)
I20260809 22:48:03.512528 7256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:03.519510 7330 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:48:03.522598 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7256
I20260809 22:48:03.523258 7330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap starting.
I20260809 22:48:03.523746 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:38655
--local_ip_for_outbound_sockets=127.21.27.129
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=35637
--webserver_interface=127.21.27.129
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:48:03.547832 7330 log.cc:824] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:03.564417 7330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:48:03.565021 7330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Bootstrap complete.
I20260809 22:48:03.574756 7330 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:48:03.575861 7330 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Initialized, Role: FOLLOWER
I20260809 22:48:03.576655 7330 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:48:03.576898 7330 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:03.577045 7330 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:03.577245 7330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 2 FOLLOWER]: Advancing to term 3
I20260809 22:48:03.581115 7330 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:48:03.581614 7330 leader_election.cc:302] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2acb266e049457eb7dfd54346eed899; no voters:
I20260809 22:48:03.582333 7330 leader_election.cc:288] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 22:48:03.582481 7334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 22:48:03.583911 7334 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [term 3 LEADER]: Becoming Leader. State: Replica: c2acb266e049457eb7dfd54346eed899, State: Running, Role: LEADER
I20260809 22:48:03.584537 7334 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } }
I20260809 22:48:03.584674 7330 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:03.587051 7336 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2acb266e049457eb7dfd54346eed899. Latest consensus state: current_term: 3 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:48:03.587503 7336 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:03.587810 7335 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c2acb266e049457eb7dfd54346eed899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2acb266e049457eb7dfd54346eed899" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42547 } } }
I20260809 22:48:03.588217 7335 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:03.596865 7343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:03.605726 7343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:03.607296 7343 catalog_manager.cc:1259] Loaded cluster ID: d1920207f5ae4a99b7bd48788a411354
I20260809 22:48:03.607460 7343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:03.611600 7343 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:03.613605 7343 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Loaded TSK: 0
I20260809 22:48:03.613755 7343 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2acb266e049457eb7dfd54346eed899: Loaded TSK: 1
F20260809 22:48:03.614884 7343 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff040d9f9ad google::LogMessage::Fail() at ??:0
@ 0x7ff040da288a google::LogMessage::SendToLog() at ??:0
@ 0x7ff040d9f4de google::LogMessage::Flush() at ??:0
@ 0x7ff040da0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff044b037d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff044af273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff0567ce1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff0567cd783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff0568592a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7ff0568590c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7ff0568a35e9 std::function<>::operator()() at ??:0
@ 0x7ff0567d1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ff0567cfc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff056857610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ff056857439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff0439eb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff0439ee91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ff0439ee799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff0439c9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff046ca36db start_thread at ??:0
@ 0x7ff03fc3961f clone at ??:0
W20260809 22:48:03.934448 7332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:03.934758 7332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:03.934857 7332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:03.942674 7332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:03.942802 7332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:03.942931 7332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:03.953230 7332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:38655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=35637
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:03.954736 7332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:03.956216 7332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:03.970288 7383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:03.970322 7380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:03.970356 7381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:03.972651 7332 server_base.cc:1034] running on GCE node
I20260809 22:48:03.973448 7332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:03.974710 7332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:03.975960 7332 hybrid_clock.cc:648] HybridClock initialized: now 1786315683975878 us; error 61 us; skew 500 ppm
I20260809 22:48:03.976373 7332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:03 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315683, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:03.979872 7332 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:48:03.981504 7332 webserver.cc:460] Webserver started at http://127.21.27.129:35637/ using document root <none> and password file <none>
I20260809 22:48:03.982317 7332 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:03.982477 7332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:03.988932 7332 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 22:48:03.992341 7390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:03.993839 7332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:03.994071 7332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "895cbc6146d44683b5390ba4ce1b2454"
format_stamp: "Formatted at 2026-08-09 22:47:17 on dist-test-slave-chsk"
server_key: "4751f5d32587a30d07fea512dca3e067"
server_key_iv: "c4c5cb2a2a5393d348dd97a31d5c7b39"
server_key_version: "encryptionkey@0"
I20260809 22:48:03.995200 7332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:04.024982 7332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:04.029371 7332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:04.029711 7332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:04.030807 7332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:04.032783 7398 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 22:48:04.038738 7332 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 22:48:04.038905 7332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 22:48:04.039077 7332 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 22:48:04.042431 7332 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 22:48:04.042546 7332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260809 22:48:04.043004 7398 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Bootstrap starting.
I20260809 22:48:04.052006 7398 log.cc:824] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:04.068717 7398 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:48:04.069823 7398 tablet_bootstrap.cc:492] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Bootstrap complete.
I20260809 22:48:04.070538 7398 ts_tablet_manager.cc:1397] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.008s
I20260809 22:48:04.082037 7398 raft_consensus.cc:348] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:04.083491 7398 raft_consensus.cc:729] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895cbc6146d44683b5390ba4ce1b2454, State: Initialized, Role: FOLLOWER
I20260809 22:48:04.084301 7398 consensus_queue.cc:260] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:04.086184 7398 ts_tablet_manager.cc:1428] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454: Time spent starting tablet: real 0.015s user 0.006s sys 0.009s
I20260809 22:48:04.087592 7332 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:38655
I20260809 22:48:04.087596 7515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:38655 every 8 connection(s)
I20260809 22:48:04.088871 7332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:04.093652 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.091695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:35893)
0809 22:48:04.092550 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.092801 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":357,"threads_started":1}
W20260809 22:48:04.094316 7516 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:42547 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:42547: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
I20260809 22:48:04.095233 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.094834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:39567)
0809 22:48:04.094995 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.095073 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:48:04.096081 7518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.095718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:54873)
0809 22:48:04.095883 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.095962 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 22:48:04.096398 7516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:48:04.096690 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7332
I20260809 22:48:04.097354 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:46065
--local_ip_for_outbound_sockets=127.21.27.130
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=46479
--webserver_interface=127.21.27.130
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:04.438069 7519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:04.438416 7519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:04.438589 7519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:04.446240 7519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:04.446417 7519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:04.446599 7519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:04.456768 7519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:46065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=46479
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:04.458129 7519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:04.459507 7519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:04.472999 7525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:04.473040 7526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:04.472962 7528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:04.473912 7519 server_base.cc:1034] running on GCE node
I20260809 22:48:04.474561 7519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:04.475600 7519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:04.476797 7519 hybrid_clock.cc:648] HybridClock initialized: now 1786315684476753 us; error 34 us; skew 500 ppm
I20260809 22:48:04.477123 7519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:04 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:04.480409 7519 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:48:04.481915 7519 webserver.cc:460] Webserver started at http://127.21.27.130:46479/ using document root <none> and password file <none>
I20260809 22:48:04.482621 7519 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:04.482774 7519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:04.488054 7519 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:04.490681 7536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.492101 7519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:04.492285 7519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "1f3ba461976f48ce9058454cbb704f5c"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "fa6da2a7f9accd492fea28548f5cb11c"
server_key_iv: "5cd521920029a31a3e9476a5fc9702af"
server_key_version: "encryptionkey@0"
I20260809 22:48:04.493264 7519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:04.504387 7519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:04.508177 7519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:04.508450 7519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:04.509395 7519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:04.510685 7519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:04.510809 7519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.510933 7519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:04.511021 7519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.545434 7519 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:46065
I20260809 22:48:04.545539 7661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:46065 every 8 connection(s)
I20260809 22:48:04.546885 7519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:04.551587 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.549797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:47897)
0809 22:48:04.550507 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.550743 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
W20260809 22:48:04.552271 7662 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:42547 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:42547: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
I20260809 22:48:04.553036 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.552716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:37785)
0809 22:48:04.552869 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.552933 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:48:04.553876 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.553504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:34653)
0809 22:48:04.553667 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.553763 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 22:48:04.554133 7662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:48:04.556612 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7519
I20260809 22:48:04.557331 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:35735
--local_ip_for_outbound_sockets=127.21.27.131
--tserver_master_addrs=127.21.27.190:42547
--webserver_port=46297
--webserver_interface=127.21.27.131
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:04.884243 7666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:04.884498 7666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:04.884595 7666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:04.891779 7666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:04.891881 7666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:04.891992 7666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:04.902557 7666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:35735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=46297
--tserver_master_addrs=127.21.27.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:04.903859 7666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:04.905172 7666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:04.918018 7675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:04.918100 7673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:04.918143 7672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:04.918989 7666 server_base.cc:1034] running on GCE node
I20260809 22:48:04.919561 7666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:04.920403 7666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:04.921593 7666 hybrid_clock.cc:648] HybridClock initialized: now 1786315684921535 us; error 49 us; skew 500 ppm
I20260809 22:48:04.921887 7666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:04 dist-test-slave-chsk krb5kdc[6073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:04.924937 7666 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:48:04.926373 7666 webserver.cc:460] Webserver started at http://127.21.27.131:46297/ using document root <none> and password file <none>
I20260809 22:48:04.927014 7666 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:04.927211 7666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:04.932257 7666 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:04.934765 7683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.936012 7666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:04.936188 7666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "eb55f31412594c92ba98ba8451e44afa"
format_stamp: "Formatted at 2026-08-09 22:47:18 on dist-test-slave-chsk"
server_key: "81b501589992dda8470cd0da33362890"
server_key_iv: "c2029543059f0a21a0c698c32e8f3622"
server_key_version: "encryptionkey@0"
I20260809 22:48:04.937142 7666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:04.952275 7666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:04.956179 7666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:04.956447 7666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:04.957326 7666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:04.958573 7666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:04.958695 7666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.958817 7666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:04.958907 7666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:04.992493 7666 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:35735
I20260809 22:48:04.992580 7808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:35735 every 8 connection(s)
I20260809 22:48:04.993959 7666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:04.995255 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7666
I20260809 22:48:04.999044 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:04.997370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:36825)
0809 22:48:04.998066 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:04.998290 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":284,"threads_started":1}
W20260809 22:48:04.999773 7809 heartbeater.cc:644] Failed to heartbeat to 127.21.27.190:42547 (0 consecutive failures): Network error: Failed to ping master at 127.21.27.190:42547: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
I20260809 22:48:05.000559 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.000267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:42081)
0809 22:48:05.000400 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.000465 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:48:05.001392 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.001118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:48961)
0809 22:48:05.001224 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.001292 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 22:48:05.001681 7809 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 22:48:05.097712 7813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.096937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:42853)
0809 22:48:05.097373 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.097451 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":210,"threads_started":1}
I20260809 22:48:05.555521 7814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.554662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:48579)
0809 22:48:05.555124 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.555257 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
I20260809 22:48:05.606303 7815 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:05.606662 7815 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:05.608374 7815 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
Aug 09 22:48:05 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315683, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
Aug 09 22:48:05 dist-test-slave-chsk krb5kdc[6073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315683, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:48:05.631667 7817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.608982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:44597 (local address 127.21.27.131:35735)
0809 22:48:05.609633 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:05.609646 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:05.609688 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:48:05.612105 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:05.612114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:05.612492 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:05.612838 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:05.614907 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.616451 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.618346 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.619052 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.619286 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:05.623720 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:05.623728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:05.623740 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:05.623777 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:05.626861 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628021 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:05.628050 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:05.628060 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:05.628260 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628842 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:05.628847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:05.628851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:05.629559 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:05.629746 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:05.630794 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0809 22:48:05.631259 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":336,"threads_started":1}
I20260809 22:48:05.631703 7819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.610104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:39453 (local address 127.21.27.130:46065)
0809 22:48:05.610815 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:05.610826 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:05.610858 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:48:05.612444 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:05.612453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:05.612758 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:05.613055 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:05.614954 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.616472 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.618343 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.618971 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.619221 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:05.623507 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:05.623516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:05.623536 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:05.623580 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:05.626845 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628021 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:05.628030 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:05.628037 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:05.628260 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628842 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:05.628847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:05.628850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:05.629566 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:05.629746 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:05.630794 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0809 22:48:05.631241 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":373,"threads_started":1}
I20260809 22:48:05.631709 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.608629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:35735 (local address 127.21.27.129:44597)
0809 22:48:05.609429 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.609481 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:05.610099 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:05.612956 (+ 2857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:05.612969 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:05.613912 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:05.614695 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:05.614711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.616679 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.616683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:05.618197 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:05.618210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.619301 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.619307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:05.619516 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:05.620418 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:05.620435 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:05.623425 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:05.627025 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:05.627029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:05.627038 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:05.627870 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:05.628394 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:05.628404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:05.628716 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:05.629706 (+ 990us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:05.629714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:05.630202 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 22:48:05.630544 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 22:48:05.631246 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":342,"spinlock_wait_cycles":35200,"thread_start_us":338,"threads_started":1}
I20260809 22:48:05.631733 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:05.609588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46065 (local address 127.21.27.129:39453)
0809 22:48:05.610215 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:05.610229 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:05.610363 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:05.613019 (+ 2656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:05.613023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:05.613906 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:05.614684 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:05.614712 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.616675 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.616679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:05.618188 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:05.618202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:05.619216 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:05.619224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:05.619509 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:05.620368 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:05.620391 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:05.623134 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:05.627012 (+ 3878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:05.627022 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:05.627038 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:05.627870 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:05.628394 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:05.628400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:05.628404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:05.628716 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:05.629706 (+ 990us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:05.629714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:05.630202 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 22:48:05.630544 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 22:48:05.631112 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":316,"spinlock_wait_cycles":27392,"thread_start_us":324,"threads_started":1}
W20260809 22:48:05.634919 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
W20260809 22:48:05.635277 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
I20260809 22:48:05.635416 7391 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454; no voters: 1f3ba461976f48ce9058454cbb704f5c, eb55f31412594c92ba98ba8451e44afa
I20260809 22:48:05.635843 7815 raft_consensus.cc:2726] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:48:06.003311 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:06.002301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:47243)
0809 22:48:06.002798 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:06.002942 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":242,"threads_started":1}
I20260809 22:48:06.099005 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:06.098591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:56807)
0809 22:48:06.098765 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:06.098863 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 22:48:06.557334 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:06.556530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:48269)
0809 22:48:06.557055 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:06.557151 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":263,"threads_started":1}
I20260809 22:48:07.004913 7822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:07.004254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:49401)
0809 22:48:07.004685 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:07.004764 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260809 22:48:07.100656 7823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:07.099934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:53713)
0809 22:48:07.100381 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:07.100473 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":221,"threads_started":1}
I20260809 22:48:07.325021 7824 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:07.325223 7824 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:07.325848 7824 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
W20260809 22:48:07.326810 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
W20260809 22:48:07.327011 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
I20260809 22:48:07.327092 7391 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454; no voters: 1f3ba461976f48ce9058454cbb704f5c, eb55f31412594c92ba98ba8451e44afa
I20260809 22:48:07.327368 7824 raft_consensus.cc:2726] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:48:07.559094 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:07.558353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:47267)
0809 22:48:07.558833 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:07.558919 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":265,"threads_started":1}
I20260809 22:48:08.006436 7826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:08.005771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:47345)
0809 22:48:08.006188 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:08.006277 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":217,"threads_started":1}
I20260809 22:48:08.102516 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:08.101646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:55667)
0809 22:48:08.102229 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:08.102333 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":288,"threads_started":1}
I20260809 22:48:08.560720 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:08.559984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:42947)
0809 22:48:08.560432 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:08.560518 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":234,"threads_started":1}
I20260809 22:48:09.008157 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:09.007422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:50139)
0809 22:48:09.007873 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:09.007955 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260809 22:48:09.104444 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:09.103633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:58001)
0809 22:48:09.104163 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:09.104261 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":247,"threads_started":1}
I20260809 22:48:09.562649 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:09.561759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:56817)
0809 22:48:09.562345 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:09.562472 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":298,"threads_started":1}
I20260809 22:48:09.958146 7832 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:09.958374 7832 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:09.959002 7832 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
W20260809 22:48:09.960076 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
W20260809 22:48:09.960305 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
I20260809 22:48:09.960439 7391 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454; no voters: 1f3ba461976f48ce9058454cbb704f5c, eb55f31412594c92ba98ba8451e44afa
I20260809 22:48:09.960716 7832 raft_consensus.cc:2726] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:48:10.009660 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:10.008989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:59015)
0809 22:48:10.009411 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:10.009487 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":243,"threads_started":1}
I20260809 22:48:10.106328 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:10.105483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:56565)
0809 22:48:10.106033 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:10.106145 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":287,"threads_started":1}
I20260809 22:48:10.564231 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:10.563501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:42613)
0809 22:48:10.563960 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:10.564054 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":260,"threads_started":1}
I20260809 22:48:11.011507 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:11.010631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:38019)
0809 22:48:11.011225 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:11.011316 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":289,"threads_started":1}
I20260809 22:48:11.108213 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:11.107387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:59033)
0809 22:48:11.107912 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:11.108003 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":285,"threads_started":1}
I20260809 22:48:11.566464 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:11.565324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:46719)
0809 22:48:11.566067 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:11.566201 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":307,"threads_started":1}
I20260809 22:48:12.013347 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:12.012519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:55629)
0809 22:48:12.013074 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:12.013164 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":267,"threads_started":1}
I20260809 22:48:12.109918 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:12.109198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:53413)
0809 22:48:12.109671 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:12.109755 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":244,"threads_started":1}
I20260809 22:48:12.568009 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:12.567353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:40801)
0809 22:48:12.567752 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:12.567842 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":203,"threads_started":1}
I20260809 22:48:13.014966 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.014255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:39219)
0809 22:48:13.014700 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.014780 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260809 22:48:13.111694 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.110813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:52819)
0809 22:48:13.111430 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.111520 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":400,"threads_started":1}
I20260809 22:48:13.569585 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:13.568851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:43963)
0809 22:48:13.569342 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:13.569427 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":230,"threads_started":1}
I20260809 22:48:14.016750 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.016017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:40067)
0809 22:48:14.016507 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:14.016590 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":248,"threads_started":1}
I20260809 22:48:14.113557 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.112701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:40067)
0809 22:48:14.113253 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:14.113334 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":311,"threads_started":1}
I20260809 22:48:14.571331 7848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:14.570518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:38561)
0809 22:48:14.571012 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:14.571093 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260809 22:48:15.002808 7849 raft_consensus.cc:482] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:15.003041 7849 raft_consensus.cc:504] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb55f31412594c92ba98ba8451e44afa" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 35735 } } peers { permanent_uuid: "1f3ba461976f48ce9058454cbb704f5c" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46065 } } peers { permanent_uuid: "895cbc6146d44683b5390ba4ce1b2454" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 38655 } }
I20260809 22:48:15.003716 7849 leader_election.cc:288] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735), 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065)
W20260809 22:48:15.004722 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eb55f31412594c92ba98ba8451e44afa (127.21.27.131:35735): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
W20260809 22:48:15.004977 7391 leader_election.cc:341] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f3ba461976f48ce9058454cbb704f5c (127.21.27.130:46065): Not found: Tablet not found: 0ee834188b07420cb7057dc0c455ed26
I20260809 22:48:15.005091 7391 leader_election.cc:302] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 895cbc6146d44683b5390ba4ce1b2454; no voters: 1f3ba461976f48ce9058454cbb704f5c, eb55f31412594c92ba98ba8451e44afa
I20260809 22:48:15.005383 7849 raft_consensus.cc:2726] T 0ee834188b07420cb7057dc0c455ed26 P 895cbc6146d44683b5390ba4ce1b2454 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:48:15.018461 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:15.017715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:33023)
0809 22:48:15.018179 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:15.018268 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260809 22:48:15.115433 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:15.114490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:52465)
0809 22:48:15.115030 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:15.115231 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":258,"threads_started":1}
I20260809 22:48:15.572911 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:15.572195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:48767)
0809 22:48:15.572641 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:15.572743 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":221,"threads_started":1}
I20260809 22:48:16.020248 7855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:16.019494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:35237)
0809 22:48:16.019981 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:16.020085 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":233,"threads_started":1}
I20260809 22:48:16.117139 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:16.116397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:59731)
0809 22:48:16.116875 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:16.116966 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":248,"threads_started":1}
I20260809 22:48:16.574648 7858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:16.573879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:42303)
0809 22:48:16.574361 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:16.574461 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":263,"threads_started":1}
I20260809 22:48:17.021970 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:17.021250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:50499)
0809 22:48:17.021714 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:17.021801 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":248,"threads_started":1}
I20260809 22:48:17.118785 7860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:17.118076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:35821)
0809 22:48:17.118527 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:17.118608 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":239,"threads_started":1}
I20260809 22:48:17.576650 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:17.575726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:39581)
0809 22:48:17.576324 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:17.576426 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":313,"threads_started":1}
I20260809 22:48:18.023744 7863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:18.022957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:33053)
0809 22:48:18.023489 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:18.023579 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":318,"threads_started":1}
I20260809 22:48:18.120616 7864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:18.119791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:48435)
0809 22:48:18.120327 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:18.120414 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":303,"threads_started":1}
I20260809 22:48:18.578577 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:18.577741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:51505)
0809 22:48:18.578257 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:18.578352 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":260,"threads_started":1}
I20260809 22:48:19.025396 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:19.024634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:33967)
0809 22:48:19.025074 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:19.025165 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260809 22:48:19.122921 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:19.122029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.129:54927)
0809 22:48:19.122651 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:19.122745 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":335,"threads_started":1}
I20260809 22:48:19.580792 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:19.579956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.130:33133)
0809 22:48:19.580486 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:19.580605 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":259,"threads_started":1}
I20260809 22:48:20.003990 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 7332
I20260809 22:48:20.024161 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:20.005266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.21.27.129:38655)
0809 22:48:20.005867 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:20.005879 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:20.005913 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:48:20.006004 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:20.006016 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:20.006248 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:20.006440 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:20.008242 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.009200 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.010447 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.011012 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.011175 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:20.017925 (+ 6750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:20.017939 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:20.017954 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:20.017989 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:20.020762 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.021629 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.021637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.021640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.021799 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.022288 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.022295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.022303 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.022696 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:20.023049 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:20.023459 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 22:48:20.023755 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":253,"threads_started":1}
I20260809 22:48:20.025626 7510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:20.028139 7871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:20.026790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:42547 (local address 127.21.27.131:52217)
0809 22:48:20.027679 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:20.027878 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.190:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":538,"threads_started":1}
I20260809 22:48:20.118732 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 7332
I20260809 22:48:20.140559 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 7519
I20260809 22:48:20.154557 7872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:20.141385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.21.27.130:46065)
0809 22:48:20.141918 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:20.141923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:20.141937 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:48:20.141991 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:20.142006 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:20.142089 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:20.142209 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:20.143815 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.144579 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.145855 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.146323 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.146461 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:20.149180 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:20.149185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:20.149189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:20.149222 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:20.151834 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.152497 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.152509 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.152512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.152643 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.153059 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.153064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.153066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.153339 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:20.153615 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:20.153904 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 22:48:20.154131 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":271,"threads_started":1}
I20260809 22:48:20.155362 7656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:20.240707 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 7519
I20260809 22:48:20.263348 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 7666
I20260809 22:48:20.277582 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:20.264253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.21.27.131:35735)
0809 22:48:20.264875 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:20.264880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:20.264898 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:48:20.264955 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:20.264959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:20.265029 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:20.265135 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:20.266560 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.267471 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.268857 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:20.269301 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:20.269400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:20.272121 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:20.272126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:20.272130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:20.272162 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:20.274972 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.275773 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.275778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.275780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.275934 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:20.276340 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:20.276344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:20.276346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:20.276629 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:20.276852 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:20.277096 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:48:20.277281 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":342,"threads_started":1}
I20260809 22:48:20.278406 7804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:20.363343 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 7666
2026-08-09T22:48:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65074 ms)
[----------] 30 tests from SecurityITest (305527 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:20 dist-test-slave-chsk krb5kdc[7880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:20 dist-test-slave-chsk krb5kdc[7880](info): set up 2 sockets
Aug 09 22:48:20 dist-test-slave-chsk krb5kdc[7880](info): commencing operation
krb5kdc: starting...
W20260809 22:48:21.551345 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:21 dist-test-slave-chsk krb5kdc[7880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:48:21.842613 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44217
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:44217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:22.174966 7894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:22.175318 7894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:22.175500 7894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:22.182782 7894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:22.182874 7894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:22.182968 7894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:22.183066 7894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:22.183145 7894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:22.192744 7894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:44217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:22.193984 7894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:22.195272 7894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:22.207261 7903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:22.209894 7901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:22.210062 7900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:22.210315 7894 server_base.cc:1034] running on GCE node
I20260809 22:48:22.211104 7894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:22.212503 7894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:22.213743 7894 hybrid_clock.cc:648] HybridClock initialized: now 1786315702213705 us; error 55 us; skew 500 ppm
I20260809 22:48:22.214064 7894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:22 dist-test-slave-chsk krb5kdc[7880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315702, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:22.217207 7894 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:48:22.218685 7894 webserver.cc:460] Webserver started at http://127.21.27.190:35989/ using document root <none> and password file <none>
I20260809 22:48:22.219401 7894 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:22.219544 7894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:22.219842 7894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:22.222723 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "dd744424f56a4f8b9f9d0ae30aabcb33"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "4a27e191b30d6821d3c97f057526c8b6"
server_key_iv: "05cf8b8eea34e23dcebf1a2521314d22"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.223524 7894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "dd744424f56a4f8b9f9d0ae30aabcb33"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "4a27e191b30d6821d3c97f057526c8b6"
server_key_iv: "05cf8b8eea34e23dcebf1a2521314d22"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.228514 7894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 22:48:22.231789 7910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:22.232955 7894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:48:22.233142 7894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "dd744424f56a4f8b9f9d0ae30aabcb33"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "4a27e191b30d6821d3c97f057526c8b6"
server_key_iv: "05cf8b8eea34e23dcebf1a2521314d22"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.233349 7894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:22.256733 7894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:22.260736 7894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:22.261044 7894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:22.288177 7894 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:44217
I20260809 22:48:22.288200 7969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:44217 every 8 connection(s)
I20260809 22:48:22.289578 7894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:22.291479 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 7894
I20260809 22:48:22.291723 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:22.292639 21614 external_mini_cluster.cc:1428] Setting key 600dcbbb9927420bf9e3552f5f0ce29c
I20260809 22:48:22.294266 7970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:48:22 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:22.308597 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: Bootstrap starting.
I20260809 22:48:22.311836 7970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:22.312841 7970 log.cc:824] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:22.316057 7970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: No bootstrap required, opened a new log
I20260809 22:48:22.316365 7973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:22.294720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.21.27.190:44217)
0809 22:48:22.295519 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:22.295530 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:22.295568 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:48:22.297320 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:22.297339 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:22.297696 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:22.298006 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:22.299821 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.301108 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.303019 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.303702 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.303823 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:22.308803 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:22.308811 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:22.308824 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:22.308862 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:22.311978 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:22.312713 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:22.312728 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:22.312734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:22.312890 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:22.313357 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:22.313370 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:22.313376 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:22.313948 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:22.314266 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:22.314677 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 22:48:22.315095 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":395,"threads_started":1}
I20260809 22:48:22.323582 7970 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } }
I20260809 22:48:22.323899 7970 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:22.323976 7970 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd744424f56a4f8b9f9d0ae30aabcb33, State: Initialized, Role: FOLLOWER
I20260809 22:48:22.324424 7970 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } }
I20260809 22:48:22.324618 7970 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:22.324715 7970 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:22.324834 7970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:22.328285 7970 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } }
I20260809 22:48:22.328933 7970 leader_election.cc:302] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd744424f56a4f8b9f9d0ae30aabcb33; no voters:
I20260809 22:48:22.329680 7970 leader_election.cc:288] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:22.329797 7976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:22.331682 7976 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [term 1 LEADER]: Becoming Leader. State: Replica: dd744424f56a4f8b9f9d0ae30aabcb33, State: Running, Role: LEADER
I20260809 22:48:22.332160 7970 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:22.332289 7976 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } }
I20260809 22:48:22.336536 7979 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd744424f56a4f8b9f9d0ae30aabcb33. Latest consensus state: current_term: 1 leader_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } } }
I20260809 22:48:22.336522 7977 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd744424f56a4f8b9f9d0ae30aabcb33" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44217 } } }
I20260809 22:48:22.336993 7979 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:22.337029 7977 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:22.338542 7985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:22.344024 7985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:22.353384 7985 catalog_manager.cc:1347] Generated new cluster ID: 0e85f25be4e64f04b5fb623d3c38d4ba
I20260809 22:48:22.353520 7985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:22.361951 7985 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:22.363814 7985 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:22.374982 7985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: Generated new TSK 0
I20260809 22:48:22.375847 7985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:48:22.455387 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:22.796663 8001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:22.796943 8001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:22.797106 8001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:22.804345 8001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:22.804448 8001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:22.804644 8001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:22.814435 8001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:22.816001 8001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:22.817322 8001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:22.830546 8006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:22.830654 8007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:22.830654 8010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:22.831732 8001 server_base.cc:1034] running on GCE node
I20260809 22:48:22.832379 8001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:22.833330 8001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:22.834601 8001 hybrid_clock.cc:648] HybridClock initialized: now 1786315702834521 us; error 72 us; skew 500 ppm
I20260809 22:48:22.834952 8001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:22 dist-test-slave-chsk krb5kdc[7880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315702, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:22.838580 8001 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:48:22.840265 8001 webserver.cc:460] Webserver started at http://127.21.27.129:33689/ using document root <none> and password file <none>
I20260809 22:48:22.841027 8001 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:22.841161 8001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:22.841506 8001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:22.844585 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "8579f6e57ede4859a873f47f59bb77f3"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "26a3b61e985fecd90d6eb0f4f6aaf851"
server_key_iv: "a85bad2b8de5b36997be982587090f1a"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.845487 8001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "8579f6e57ede4859a873f47f59bb77f3"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "26a3b61e985fecd90d6eb0f4f6aaf851"
server_key_iv: "a85bad2b8de5b36997be982587090f1a"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.850790 8001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:48:22.854312 8017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:22.855711 8001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:22.855890 8001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "8579f6e57ede4859a873f47f59bb77f3"
format_stamp: "Formatted at 2026-08-09 22:48:22 on dist-test-slave-chsk"
server_key: "26a3b61e985fecd90d6eb0f4f6aaf851"
server_key_iv: "a85bad2b8de5b36997be982587090f1a"
server_key_version: "encryptionkey@0"
I20260809 22:48:22.856098 8001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:22.869429 8001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:22.873888 8001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:22.874199 8001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:22.875277 8001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:22.876784 8001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:22.876889 8001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:22.877035 8001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:22.877112 8001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:22.911315 8001 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:45799
I20260809 22:48:22.911357 8142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:45799 every 8 connection(s)
I20260809 22:48:22.912613 8001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:22.917310 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8001
I20260809 22:48:22.917567 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:22.918491 21614 external_mini_cluster.cc:1428] Setting key 0c899c34b275c6f327449adedc80d27b
Aug 09 22:48:22 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315702, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:22.936029 8147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:22.915230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:35445 (local address 127.21.27.190:44217)
0809 22:48:22.915714 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:22.915719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:22.916643 (+ 924us) server_negotiation.cc:407] Connection header received
0809 22:48:22.919428 (+ 2785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:22.919434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:22.919527 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:22.919666 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:22.921507 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.922235 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.924796 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.925194 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.925291 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:22.929859 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:22.929864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:22.929867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:22.929901 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:22.932645 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:22.933457 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:22.933462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:22.933466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:22.933620 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:22.934141 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:22.934146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:22.934150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:22.934440 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:22.934607 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:22.935381 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 22:48:22.935613 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":222,"threads_started":1}
I20260809 22:48:22.937076 8148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:22.915767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44217 (local address 127.21.27.129:35445)
0809 22:48:22.916480 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:22.916518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:22.919089 (+ 2571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:22.919914 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:22.919923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:22.920467 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:22.921306 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:22.921332 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.922395 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.922403 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:22.924653 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:22.924671 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:22.925431 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:22.925441 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:22.925765 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:22.926651 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:22.926684 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:22.929593 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:22.932857 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:22.932867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:22.932885 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:22.933320 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:22.933747 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:22.933752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:22.933755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:22.934020 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:22.934581 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:22.934588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:22.934950 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 22:48:22.935270 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 22:48:22.935838 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":286,"threads_started":1}
I20260809 22:48:22.938731 8144 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44217
I20260809 22:48:22.939080 8144 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:22.940008 8144 heartbeater.cc:511] Master 127.21.27.190:44217 requested a full tablet report, sending...
I20260809 22:48:22.942642 7929 ts_manager.cc:194] Registered new tserver with Master: 8579f6e57ede4859a873f47f59bb77f3 (127.21.27.129:45799)
I20260809 22:48:22.944962 7929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:35445
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:48:22.988821 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:23.329270 8151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:23.329566 8151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:23.329658 8151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:23.337275 8151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:23.337396 8151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:23.337518 8151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:23.347816 8151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:23.349083 8151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:23.350663 8151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:23.364087 8160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:23.364171 8157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:23.364166 8158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:23.365302 8151 server_base.cc:1034] running on GCE node
I20260809 22:48:23.365917 8151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:23.367237 8151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:23.368546 8151 hybrid_clock.cc:648] HybridClock initialized: now 1786315703368481 us; error 50 us; skew 500 ppm
I20260809 22:48:23.368841 8151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:23 dist-test-slave-chsk krb5kdc[7880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:23.372017 8151 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:48:23.373575 8151 webserver.cc:460] Webserver started at http://127.21.27.130:36273/ using document root <none> and password file <none>
I20260809 22:48:23.374284 8151 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:23.374441 8151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:23.374781 8151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:23.377872 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "34b4b9dab7cc44e89ee3d21fbba3b590"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fcfa03ebb3f78c76c762ac816a0a35f3"
server_key_iv: "97975c795a320c0e65557722ac093d52"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.378683 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "34b4b9dab7cc44e89ee3d21fbba3b590"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fcfa03ebb3f78c76c762ac816a0a35f3"
server_key_iv: "97975c795a320c0e65557722ac093d52"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.384173 8151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:48:23.387892 8168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.389189 8151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:23.389382 8151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "34b4b9dab7cc44e89ee3d21fbba3b590"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fcfa03ebb3f78c76c762ac816a0a35f3"
server_key_iv: "97975c795a320c0e65557722ac093d52"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.389609 8151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:23.418025 8151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:23.422286 8151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:23.422605 8151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:23.423679 8151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:23.424979 8151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:23.425113 8151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.425268 8151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:23.425344 8151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.460449 8151 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:34813
I20260809 22:48:23.460515 8295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:34813 every 8 connection(s)
I20260809 22:48:23.461781 8151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:23.468848 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8151
I20260809 22:48:23.469051 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:23.469640 21614 external_mini_cluster.cc:1428] Setting key d6d029c199dda65ced4886ab40201fd9
Aug 09 22:48:23 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:23.484720 8299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:23.464539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:37143 (local address 127.21.27.190:44217)
0809 22:48:23.465007 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:23.465012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:23.465942 (+ 930us) server_negotiation.cc:407] Connection header received
0809 22:48:23.467834 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:23.467840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:23.467933 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:23.468053 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:23.470145 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.471062 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.473281 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.473736 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.473859 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:23.478535 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:23.478540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:23.478543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:23.478578 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:23.481395 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:23.482213 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:23.482223 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:23.482226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:23.482370 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:23.482841 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:23.482845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:23.482847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:23.483197 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:23.483372 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:23.484084 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 22:48:23.484335 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":227,"threads_started":1}
I20260809 22:48:23.485816 8300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:23.465112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44217 (local address 127.21.27.130:37143)
0809 22:48:23.465784 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:23.465819 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:23.467567 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:23.468304 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:23.468312 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:23.469147 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:23.469951 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:23.469977 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.471251 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.471256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:23.473106 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:23.473126 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.474001 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.474010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:23.474361 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:23.475233 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:23.475275 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:23.478256 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:23.481561 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:23.481571 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:23.481589 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:23.482065 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:23.482493 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:23.482498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:23.482501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:23.482710 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:23.483330 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:23.483339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:23.483718 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 22:48:23.483992 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 22:48:23.484544 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":266,"threads_started":1}
I20260809 22:48:23.487552 8296 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44217
I20260809 22:48:23.487887 8296 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:23.488647 8296 heartbeater.cc:511] Master 127.21.27.190:44217 requested a full tablet report, sending...
I20260809 22:48:23.490238 7929 ts_manager.cc:194] Registered new tserver with Master: 34b4b9dab7cc44e89ee3d21fbba3b590 (127.21.27.130:34813)
I20260809 22:48:23.491565 7929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:37143
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:48:23.537784 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:23.872843 8303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:23.873103 8303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:23.873270 8303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:23.880300 8303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:23.880429 8303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:23.880616 8303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:23.890187 8303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:23.891467 8303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:23.892717 8303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:23.905683 8308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:23.905674 8311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:23.905728 8309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:23.906448 8303 server_base.cc:1034] running on GCE node
I20260809 22:48:23.907044 8303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:23.907981 8303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:23.909214 8303 hybrid_clock.cc:648] HybridClock initialized: now 1786315703909162 us; error 44 us; skew 500 ppm
I20260809 22:48:23.909524 8303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:23 dist-test-slave-chsk krb5kdc[7880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:23.912844 8303 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:48:23.914350 8303 webserver.cc:460] Webserver started at http://127.21.27.131:39121/ using document root <none> and password file <none>
I20260809 22:48:23.915088 8303 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:23.915307 8303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:23.915616 8303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:23.918358 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "068b2bedbab2480cac9eb914b304ebdb"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fabf18fe4d3b7edab37fae724ebef6ab"
server_key_iv: "3211de22c948101fcc24893d4daba641"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.919106 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "068b2bedbab2480cac9eb914b304ebdb"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fabf18fe4d3b7edab37fae724ebef6ab"
server_key_iv: "3211de22c948101fcc24893d4daba641"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.924122 8303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:48:23.927488 8319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.928610 8303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260809 22:48:23.928817 8303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "068b2bedbab2480cac9eb914b304ebdb"
format_stamp: "Formatted at 2026-08-09 22:48:23 on dist-test-slave-chsk"
server_key: "fabf18fe4d3b7edab37fae724ebef6ab"
server_key_iv: "3211de22c948101fcc24893d4daba641"
server_key_version: "encryptionkey@0"
I20260809 22:48:23.929024 8303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:23.942452 8303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:23.946462 8303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:23.946764 8303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:23.947825 8303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:23.948971 8144 heartbeater.cc:503] Master 127.21.27.190:44217 was elected leader, sending a full tablet report...
I20260809 22:48:23.949151 8303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:23.949250 8303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.949386 8303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:23.949455 8303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:23.983978 8303 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:41847
I20260809 22:48:23.984016 8440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:41847 every 8 connection(s)
I20260809 22:48:23.985262 8303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:23.986872 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8303
I20260809 22:48:23.987103 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:23.987692 21614 external_mini_cluster.cc:1428] Setting key d09532d4671154f0995584586494dc81
Aug 09 22:48:23 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:24.006526 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:23.988254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:44037 (local address 127.21.27.190:44217)
0809 22:48:23.988772 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:23.988777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:23.989973 (+ 1196us) server_negotiation.cc:407] Connection header received
0809 22:48:23.992122 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:23.992127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:23.992219 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:23.992336 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:23.994106 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.994929 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.996517 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.996926 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.997069 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.000542 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:24.000547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:24.000550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:24.000583 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:24.003452 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.004189 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.004199 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.004201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.004317 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.004760 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.004764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.004769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.005067 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:24.005228 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.005916 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.006148 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":266,"threads_started":1}
I20260809 22:48:24.007494 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:23.988802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44217 (local address 127.21.27.131:44037)
0809 22:48:23.989646 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:23.989689 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:23.991851 (+ 2162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:23.992594 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:23.992602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:23.993318 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:23.993882 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:23.993899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.995089 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.995094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:23.996373 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:23.996385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:23.997156 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:23.997164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:23.997407 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:23.998102 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:23.998124 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:24.000334 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:24.003606 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:24.003615 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:24.003631 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:24.004055 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:24.004429 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:24.004434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:24.004438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:24.004639 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:24.005201 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:24.005208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:24.005557 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.005810 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.006324 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":373,"threads_started":1}
I20260809 22:48:24.008992 8441 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44217
I20260809 22:48:24.009352 8441 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:24.010048 8441 heartbeater.cc:511] Master 127.21.27.190:44217 requested a full tablet report, sending...
I20260809 22:48:24.011497 7929 ts_manager.cc:194] Registered new tserver with Master: 068b2bedbab2480cac9eb914b304ebdb (127.21.27.131:41847)
I20260809 22:48:24.012262 7929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:44037
I20260809 22:48:24.015892 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:24.033525 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.020534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.21.27.190:44217)
0809 22:48:24.020701 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.020705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.021231 (+ 526us) server_negotiation.cc:407] Connection header received
0809 22:48:24.021434 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.021439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.021515 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.021619 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:24.023101 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.023872 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.025180 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.025576 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.025730 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.028149 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:24.028155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:24.028160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:24.028199 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:24.030687 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.031416 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.031423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.031426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.031558 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.031933 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.031941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.031945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.032324 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:24.032566 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.032929 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.033155 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:48:24.037498 7929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:24.044911 7929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:24.068365 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.057282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.21.27.131:41847)
0809 22:48:24.059213 (+ 1931us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.059223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.059256 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:48:24.059356 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.059364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.059603 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.060703 (+ 1100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:24.060996 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.062642 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.065472 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.067056 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.067230 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.067464 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.067801 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.067982 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1708,"thread_start_us":1549,"threads_started":1}
I20260809 22:48:24.068423 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.056922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:41847 (local address 127.0.0.1:40928)
0809 22:48:24.057753 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:24.057796 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:24.058021 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:24.059857 (+ 1836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:24.059866 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:24.059897 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:24.060566 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.060584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.062805 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.062810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.065309 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.065321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.067276 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.067283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.067523 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.067585 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.067734 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.067919 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":149,"thread_start_us":405,"threads_started":1}
I20260809 22:48:24.069140 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.058756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.21.27.130:34813)
0809 22:48:24.059383 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.059391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.059413 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:48:24.059496 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.059503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.059700 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.060216 (+ 516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:24.060975 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.062643 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.065615 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.067063 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.067230 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.067427 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.068784 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.068932 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":776,"threads_started":1}
I20260809 22:48:24.069399 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.057506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:34813 (local address 127.0.0.1:45062)
0809 22:48:24.058735 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:24.058755 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:24.058869 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:24.059880 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:24.059883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:24.059923 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:24.060767 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.060782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.062846 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.062854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.065454 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.065469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.067615 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.067619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.067820 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.067851 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.067940 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.068062 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":108,"spinlock_wait_cycles":2944,"thread_start_us":980,"threads_started":1}
I20260809 22:48:24.071635 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.058311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45799 (local address 127.0.0.1:60110)
0809 22:48:24.063157 (+ 4846us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:24.063181 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:24.063316 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:24.064001 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:24.064012 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:24.064042 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:24.064527 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.064541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.066336 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.066341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.068672 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.068686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.070847 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.070855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.071017 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.071048 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.071178 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.071306 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4675,"thread_start_us":4451,"threads_started":1}
I20260809 22:48:24.071753 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.059346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.21.27.129:45799)
0809 22:48:24.060117 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.060128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.063445 (+ 3317us) server_negotiation.cc:407] Connection header received
0809 22:48:24.063547 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.063555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.063803 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.063991 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:24.064741 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.066179 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.068863 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.070319 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.070455 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.070704 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.071240 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.071425 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":375,"threads_started":1}
I20260809 22:48:24.072661 8368 tablet_service.cc:1467] Processing CreateTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (DEFAULT_TABLE table=test-table [id=26323197116f4885ad37c40e3ad50435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:24.072849 8220 tablet_service.cc:1467] Processing CreateTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (DEFAULT_TABLE table=test-table [id=26323197116f4885ad37c40e3ad50435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:24.073982 8220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01146dc0fc1d49448cfd287f2e9ff3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:24.073983 8368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01146dc0fc1d49448cfd287f2e9ff3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:24.083213 8071 tablet_service.cc:1467] Processing CreateTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (DEFAULT_TABLE table=test-table [id=26323197116f4885ad37c40e3ad50435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:24.084776 8071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01146dc0fc1d49448cfd287f2e9ff3b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:24.087142 8462 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Bootstrap starting.
I20260809 22:48:24.089941 8462 tablet_bootstrap.cc:654] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:24.090829 8463 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Bootstrap starting.
I20260809 22:48:24.091332 8462 log.cc:824] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:24.094655 8463 tablet_bootstrap.cc:654] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:24.095640 8462 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: No bootstrap required, opened a new log
I20260809 22:48:24.095934 8462 ts_tablet_manager.cc:1397] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 22:48:24.096050 8463 log.cc:824] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:24.099310 8463 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: No bootstrap required, opened a new log
I20260809 22:48:24.099665 8463 ts_tablet_manager.cc:1397] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260809 22:48:24.104682 8467 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Bootstrap starting.
I20260809 22:48:24.106722 8462 raft_consensus.cc:348] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.107208 8462 raft_consensus.cc:374] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:24.107358 8462 raft_consensus.cc:729] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068b2bedbab2480cac9eb914b304ebdb, State: Initialized, Role: FOLLOWER
I20260809 22:48:24.108078 8467 tablet_bootstrap.cc:654] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:24.107981 8462 consensus_queue.cc:260] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.109160 8441 heartbeater.cc:503] Master 127.21.27.190:44217 was elected leader, sending a full tablet report...
I20260809 22:48:24.109408 8467 log.cc:824] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:24.110365 8463 raft_consensus.cc:348] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.110859 8463 raft_consensus.cc:374] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:24.111020 8463 raft_consensus.cc:729] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b4b9dab7cc44e89ee3d21fbba3b590, State: Initialized, Role: FOLLOWER
I20260809 22:48:24.111056 8462 ts_tablet_manager.cc:1428] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260809 22:48:24.111722 8463 consensus_queue.cc:260] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.112650 8467 tablet_bootstrap.cc:492] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: No bootstrap required, opened a new log
I20260809 22:48:24.112928 8467 ts_tablet_manager.cc:1397] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:48:24.113426 8296 heartbeater.cc:503] Master 127.21.27.190:44217 was elected leader, sending a full tablet report...
I20260809 22:48:24.113485 8463 ts_tablet_manager.cc:1428] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260809 22:48:24.121287 8467 raft_consensus.cc:348] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.121670 8467 raft_consensus.cc:374] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:24.121773 8467 raft_consensus.cc:729] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8579f6e57ede4859a873f47f59bb77f3, State: Initialized, Role: FOLLOWER
I20260809 22:48:24.122423 8467 consensus_queue.cc:260] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.123916 8467 ts_tablet_manager.cc:1428] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
W20260809 22:48:24.165016 8146 tablet.cc:2369] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:24.167053 8471 raft_consensus.cc:482] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:24.167362 8471 raft_consensus.cc:504] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.168911 8471 leader_election.cc:288] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068b2bedbab2480cac9eb914b304ebdb (127.21.27.131:41847), 8579f6e57ede4859a873f47f59bb77f3 (127.21.27.129:45799)
I20260809 22:48:24.176071 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.170353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:42665 (local address 127.21.27.129:45799)
0809 22:48:24.170552 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.170559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.170874 (+ 315us) server_negotiation.cc:407] Connection header received
0809 22:48:24.171001 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.171006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.171087 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.171233 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:24.171779 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.172464 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.174261 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.175076 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.175291 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.175368 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.175678 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.175807 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:48:24.176079 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.169347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:48241 (local address 127.21.27.131:41847)
0809 22:48:24.169560 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.169565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.170233 (+ 668us) server_negotiation.cc:407] Connection header received
0809 22:48:24.170466 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.170473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.170554 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.170628 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:24.171568 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.172301 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.174201 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.175003 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.175215 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.175297 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.175661 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.175789 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:48:24.176292 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.170142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:45799 (local address 127.21.27.130:42665)
0809 22:48:24.170761 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:24.170775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:24.170880 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:24.171249 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:24.171252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:24.171272 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:24.171624 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.171632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.172645 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.172649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.174098 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.174110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.175326 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.175333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.175526 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.175549 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.175653 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.175789 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":9,"thread_start_us":320,"threads_started":1}
I20260809 22:48:24.176321 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.169368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:41847 (local address 127.21.27.130:48241)
0809 22:48:24.170058 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:24.170107 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:24.170279 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:24.170839 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:24.170846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:24.170942 (+ 96us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:24.171421 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.171433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.172429 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.172437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.174027 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:24.174039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.175265 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.175271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:24.175488 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.175526 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:48:24.175633 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:48:24.175788 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":72,"spinlock_wait_cycles":21760,"thread_start_us":382,"threads_started":1}
I20260809 22:48:24.177431 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01146dc0fc1d49448cfd287f2e9ff3b9" candidate_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068b2bedbab2480cac9eb914b304ebdb" is_pre_election: true
I20260809 22:48:24.177429 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01146dc0fc1d49448cfd287f2e9ff3b9" candidate_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8579f6e57ede4859a873f47f59bb77f3" is_pre_election: true
I20260809 22:48:24.177932 8389 raft_consensus.cc:2454] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b4b9dab7cc44e89ee3d21fbba3b590 in term 0.
I20260809 22:48:24.177935 8092 raft_consensus.cc:2454] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34b4b9dab7cc44e89ee3d21fbba3b590 in term 0.
I20260809 22:48:24.178599 8170 leader_election.cc:302] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34b4b9dab7cc44e89ee3d21fbba3b590, 8579f6e57ede4859a873f47f59bb77f3; no voters:
I20260809 22:48:24.178985 8471 raft_consensus.cc:2781] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:24.179127 8471 raft_consensus.cc:482] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:24.179271 8471 raft_consensus.cc:3037] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:24.181524 8471 raft_consensus.cc:504] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.182149 8471 leader_election.cc:288] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [CANDIDATE]: Term 1 election: Requested vote from peers 068b2bedbab2480cac9eb914b304ebdb (127.21.27.131:41847), 8579f6e57ede4859a873f47f59bb77f3 (127.21.27.129:45799)
I20260809 22:48:24.182672 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01146dc0fc1d49448cfd287f2e9ff3b9" candidate_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068b2bedbab2480cac9eb914b304ebdb"
I20260809 22:48:24.182874 8389 raft_consensus.cc:3037] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:24.182864 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01146dc0fc1d49448cfd287f2e9ff3b9" candidate_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8579f6e57ede4859a873f47f59bb77f3"
I20260809 22:48:24.183133 8092 raft_consensus.cc:3037] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:24.185369 8389 raft_consensus.cc:2454] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b4b9dab7cc44e89ee3d21fbba3b590 in term 1.
I20260809 22:48:24.185757 8170 leader_election.cc:302] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068b2bedbab2480cac9eb914b304ebdb, 34b4b9dab7cc44e89ee3d21fbba3b590; no voters:
I20260809 22:48:24.185827 8092 raft_consensus.cc:2454] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34b4b9dab7cc44e89ee3d21fbba3b590 in term 1.
I20260809 22:48:24.186023 8471 raft_consensus.cc:2781] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:24.186524 8471 raft_consensus.cc:686] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 1 LEADER]: Becoming Leader. State: Replica: 34b4b9dab7cc44e89ee3d21fbba3b590, State: Running, Role: LEADER
I20260809 22:48:24.187060 8471 consensus_queue.cc:237] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } }
I20260809 22:48:24.191736 7929 catalog_manager.cc:5495] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34b4b9dab7cc44e89ee3d21fbba3b590 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34b4b9dab7cc44e89ee3d21fbba3b590" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34813 } health_report { overall_health: HEALTHY } } }
W20260809 22:48:24.213886 8298 tablet.cc:2369] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:24.230456 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.225157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.21.27.130:34813)
0809 22:48:24.225344 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.225349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.225501 (+ 152us) server_negotiation.cc:407] Connection header received
0809 22:48:24.225682 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.225686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.225775 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.225901 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:24.226275 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.226951 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.228473 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.228885 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.228978 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.229455 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:24.229953 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:24.230058 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.230165 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.230235 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 22:48:24.236994 8442 tablet.cc:2369] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:24.239179 8389 raft_consensus.cc:1261] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 1 FOLLOWER]: Refusing update from remote peer 34b4b9dab7cc44e89ee3d21fbba3b590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:24.239177 8092 raft_consensus.cc:1261] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 1 FOLLOWER]: Refusing update from remote peer 34b4b9dab7cc44e89ee3d21fbba3b590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:24.240247 8475 consensus_queue.cc:1035] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068b2bedbab2480cac9eb914b304ebdb" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 41847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:24.240653 8471 consensus_queue.cc:1035] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8579f6e57ede4859a873f47f59bb77f3" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 45799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:24.257413 8479 mvcc.cc:205] Tried to move back new op lower bound from 7316749124552712192 to 7316749124354928640. Current Snapshot: MvccSnapshot[applied={T|T < 7316749124552712192 or (T in {7316749124552712192})}]
I20260809 22:48:24.257809 8480 mvcc.cc:205] Tried to move back new op lower bound from 7316749124552712192 to 7316749124354928640. Current Snapshot: MvccSnapshot[applied={T|T < 7316749124552712192}]
I20260809 22:48:24.266559 7929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:24.266842 7929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:24.270471 7929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd744424f56a4f8b9f9d0ae30aabcb33: Sending DeleteTablet for 3 replicas of tablet 01146dc0fc1d49448cfd287f2e9ff3b9
I20260809 22:48:24.271704 8220 tablet_service.cc:1514] Processing DeleteTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:24 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:45062
I20260809 22:48:24.271996 8071 tablet_service.cc:1514] Processing DeleteTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:24 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:60110
I20260809 22:48:24.272004 8368 tablet_service.cc:1514] Processing DeleteTablet for tablet 01146dc0fc1d49448cfd287f2e9ff3b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:24 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:40928
I20260809 22:48:24.274457 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8001
I20260809 22:48:24.284754 8489 tablet_replica.cc:330] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: stopping tablet replica
I20260809 22:48:24.285499 8489 raft_consensus.cc:2229] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:24.286476 8489 raft_consensus.cc:2258] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:24.288189 8492 tablet_replica.cc:330] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: stopping tablet replica
I20260809 22:48:24.288687 8492 raft_consensus.cc:2229] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:24.289120 8489 ts_tablet_manager.cc:1905] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:24.289206 8492 raft_consensus.cc:2258] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:24.291209 8492 ts_tablet_manager.cc:1905] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:24.295154 8491 tablet_replica.cc:330] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: stopping tablet replica
I20260809 22:48:24.295666 8491 raft_consensus.cc:2229] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:24.298542 8489 ts_tablet_manager.cc:1918] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:24.298772 8489 log.cc:1196] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/01146dc0fc1d49448cfd287f2e9ff3b9
I20260809 22:48:24.299286 8491 raft_consensus.cc:2258] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:24.299494 8489 ts_tablet_manager.cc:1939] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 34b4b9dab7cc44e89ee3d21fbba3b590: Deleting consensus metadata
I20260809 22:48:24.301757 8491 ts_tablet_manager.cc:1905] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:24.301874 7911 catalog_manager.cc:4841] TS 34b4b9dab7cc44e89ee3d21fbba3b590 (127.21.27.130:34813): tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (table test-table [id=26323197116f4885ad37c40e3ad50435]) successfully deleted
I20260809 22:48:24.303426 8492 ts_tablet_manager.cc:1918] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:24.303599 8492 log.cc:1196] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/01146dc0fc1d49448cfd287f2e9ff3b9
I20260809 22:48:24.304122 8492 ts_tablet_manager.cc:1939] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 8579f6e57ede4859a873f47f59bb77f3: Deleting consensus metadata
I20260809 22:48:24.305958 7912 catalog_manager.cc:4841] TS 8579f6e57ede4859a873f47f59bb77f3 (127.21.27.129:45799): tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (table test-table [id=26323197116f4885ad37c40e3ad50435]) successfully deleted
I20260809 22:48:24.312460 8491 ts_tablet_manager.cc:1918] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:24.312703 8491 log.cc:1196] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/01146dc0fc1d49448cfd287f2e9ff3b9
I20260809 22:48:24.313264 8491 ts_tablet_manager.cc:1939] T 01146dc0fc1d49448cfd287f2e9ff3b9 P 068b2bedbab2480cac9eb914b304ebdb: Deleting consensus metadata
Aug 09 22:48:24 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:48:24.315078 7912 catalog_manager.cc:4841] TS 068b2bedbab2480cac9eb914b304ebdb (127.21.27.131:41847): tablet 01146dc0fc1d49448cfd287f2e9ff3b9 (table test-table [id=26323197116f4885ad37c40e3ad50435]) successfully deleted
I20260809 22:48:24.323452 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.281521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.21.27.129:45799)
0809 22:48:24.281747 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.281756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.287554 (+ 5798us) server_negotiation.cc:407] Connection header received
0809 22:48:24.295246 (+ 7692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.295254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.295347 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.295473 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:24.297594 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.298772 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.300853 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.301482 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.301580 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.316183 (+ 14603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:24.316207 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:24.316226 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:24.316280 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:24.320214 (+ 3934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.321181 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.321189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.321191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.321334 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.321751 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.321754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.321756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.322132 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:24.322357 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.322830 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.323008 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:48:24.324096 8138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:24.419104 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8001
I20260809 22:48:24.440778 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8151
Aug 09 22:48:24 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:48:24.459004 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.441770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.21.27.130:34813)
0809 22:48:24.441980 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.441986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.442019 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:48:24.442088 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.442094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.442186 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.442304 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:24.443600 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.444903 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.446129 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.446783 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.446916 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.451751 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:24.451770 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:24.451790 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:24.451840 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:24.455955 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.456723 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.456733 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.456737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.456958 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.457350 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.457357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.457364 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.457847 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:24.458213 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.458359 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.458593 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:48:24.459868 8291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:24.576611 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8151
I20260809 22:48:24.596812 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8303
Aug 09 22:48:24 dist-test-slave-chsk krb5kdc[7880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:48:24.612159 8456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:24.597720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.21.27.131:41847)
0809 22:48:24.597897 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:24.597902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:24.597921 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:48:24.598093 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:24.598097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:24.598162 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:24.598248 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:24.599662 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.600397 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.601641 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:24.602030 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:24.602140 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:24.606737 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:24.606749 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:24.606764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:24.606794 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:24.609628 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.610226 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.610233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.610235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.610387 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:24.610715 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:24.610718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:24.610720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:24.611072 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:24.611398 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:24.611671 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 22:48:24.611872 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:48:24.612793 8435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:24.703022 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8303
I20260809 22:48:24.721894 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 7894
I20260809 22:48:24.723119 7965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:24.832737 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 7894
2026-08-09T22:48:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4476 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T22:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:24Z Disabled control of system clock
I20260809 22:48:24.924008 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44537
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:33505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:44537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:25.260232 8503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:25.260511 8503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:25.260665 8503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:25.268007 8503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:25.268138 8503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:25.268214 8503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:25.268311 8503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:25.268391 8503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:25.277783 8503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:44537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:44537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:25.279479 8503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:25.281889 8503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:25.294047 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:25.294070 8513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:25.294252 8510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:25.294708 8503 server_base.cc:1034] running on GCE node
I20260809 22:48:25.295606 8503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:25.297020 8503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:25.298270 8503 hybrid_clock.cc:648] HybridClock initialized: now 1786315705298225 us; error 49 us; skew 500 ppm
I20260809 22:48:25.298617 8503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:25.307760 8503 webserver.cc:460] Webserver started at http://127.21.27.190:43685/ using document root <none> and password file <none>
I20260809 22:48:25.308485 8503 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:25.308630 8503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:25.309003 8503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:25.312106 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "6714b3059bb14d84aab501222dbef3ee"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "cdfdf81a65dfdd4e76c73da930685c84"
server_key_iv: "21d2e001dcb7635ba06f252354b3a1cf"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.312945 8503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "6714b3059bb14d84aab501222dbef3ee"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "cdfdf81a65dfdd4e76c73da930685c84"
server_key_iv: "21d2e001dcb7635ba06f252354b3a1cf"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.318243 8503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:48:25.321658 8520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:25.322939 8503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:48:25.323199 8503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "6714b3059bb14d84aab501222dbef3ee"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "cdfdf81a65dfdd4e76c73da930685c84"
server_key_iv: "21d2e001dcb7635ba06f252354b3a1cf"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.323424 8503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:25.336274 8503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:25.337039 8503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:25.337360 8503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:25.356987 8503 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:44537
I20260809 22:48:25.357007 8577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:44537 every 8 connection(s)
I20260809 22:48:25.358361 8503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:25.360179 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8503
I20260809 22:48:25.360373 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:25.361001 21614 external_mini_cluster.cc:1428] Setting key e7d7d2304ff5f7645ced17831a4276ae
I20260809 22:48:25.363093 8578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:25.374971 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:25.362909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.21.27.190:44537)
0809 22:48:25.364043 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:25.364055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:25.364095 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:48:25.365761 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:25.365782 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:25.366220 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:25.366555 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:25.367242 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.368629 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.370526 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.371255 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.371382 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:25.372669 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:25.372676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:25.372688 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:25.373285 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:25.373368 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:25.373388 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:25.373471 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:25.373730 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 22:48:25.374066 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":508,"threads_started":1}
I20260809 22:48:25.376173 8578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: Bootstrap starting.
I20260809 22:48:25.379202 8578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:25.380194 8578 log.cc:824] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:25.383104 8578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: No bootstrap required, opened a new log
I20260809 22:48:25.390894 8578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } }
I20260809 22:48:25.391311 8578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:25.391392 8578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6714b3059bb14d84aab501222dbef3ee, State: Initialized, Role: FOLLOWER
I20260809 22:48:25.391924 8578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } }
I20260809 22:48:25.392125 8578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:25.392230 8578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:25.392354 8578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:25.394856 8578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } }
I20260809 22:48:25.395383 8578 leader_election.cc:302] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6714b3059bb14d84aab501222dbef3ee; no voters:
I20260809 22:48:25.395896 8578 leader_election.cc:288] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:25.396054 8583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:25.396801 8583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [term 1 LEADER]: Becoming Leader. State: Replica: 6714b3059bb14d84aab501222dbef3ee, State: Running, Role: LEADER
I20260809 22:48:25.397328 8583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } }
I20260809 22:48:25.397989 8578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:25.400362 8585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6714b3059bb14d84aab501222dbef3ee. Latest consensus state: current_term: 1 leader_uuid: "6714b3059bb14d84aab501222dbef3ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } } }
I20260809 22:48:25.400563 8584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6714b3059bb14d84aab501222dbef3ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6714b3059bb14d84aab501222dbef3ee" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 44537 } } }
I20260809 22:48:25.400897 8584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:25.400904 8585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:25.403733 8594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:25.408324 8594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:25.416397 8594 catalog_manager.cc:1347] Generated new cluster ID: dd1be0a5b3f5444e949aeef7aca704cb
I20260809 22:48:25.416523 8594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:25.436628 8594 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:25.437964 8594 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:25.450333 8594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: Generated new TSK 0
I20260809 22:48:25.451360 8594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:25.458371 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--builtin_ntp_servers=127.21.27.148:33505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:25.802039 8604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:25.802342 8604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:25.802508 8604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:25.809731 8604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:25.809875 8604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:25.810065 8604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:25.819170 8604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:25.820497 8604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:25.822305 8604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:25.836050 8610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:25.836130 8613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:25.836159 8611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:25.837217 8604 server_base.cc:1034] running on GCE node
I20260809 22:48:25.837836 8604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:25.838791 8604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:25.840034 8604 hybrid_clock.cc:648] HybridClock initialized: now 1786315705839992 us; error 50 us; skew 500 ppm
I20260809 22:48:25.840368 8604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:25.842684 8604 webserver.cc:460] Webserver started at http://127.21.27.129:36803/ using document root <none> and password file <none>
I20260809 22:48:25.843464 8604 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:25.843603 8604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:25.843927 8604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:25.846829 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "df064aab18384635afffbcb462f30030"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "14fe074fee4b659a4637968100f52b1e"
server_key_iv: "3a2321e0f9d78e5a2d754518e433f171"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.847697 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "df064aab18384635afffbcb462f30030"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "14fe074fee4b659a4637968100f52b1e"
server_key_iv: "3a2321e0f9d78e5a2d754518e433f171"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.853029 8604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:48:25.856541 8619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:25.857843 8604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 22:48:25.858013 8604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "df064aab18384635afffbcb462f30030"
format_stamp: "Formatted at 2026-08-09 22:48:25 on dist-test-slave-chsk"
server_key: "14fe074fee4b659a4637968100f52b1e"
server_key_iv: "3a2321e0f9d78e5a2d754518e433f171"
server_key_version: "encryptionkey@0"
I20260809 22:48:25.858232 8604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:25.873575 8604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:25.874361 8604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:25.874675 8604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:25.875780 8604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:25.877213 8604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:25.877338 8604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:25.877466 8604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:25.877552 8604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:25.911504 8604 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:37897
I20260809 22:48:25.911597 8743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:37897 every 8 connection(s)
I20260809 22:48:25.912833 8604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:25.920722 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8604
I20260809 22:48:25.920923 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:25.921512 21614 external_mini_cluster.cc:1428] Setting key 3ed42d65c4614fb06c1dbcab2adf0134
I20260809 22:48:25.924324 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--builtin_ntp_servers=127.21.27.148:33505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:25.925681 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:25.915301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:43039 (local address 127.21.27.190:44537)
0809 22:48:25.915761 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:25.915769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:25.916982 (+ 1213us) server_negotiation.cc:407] Connection header received
0809 22:48:25.918639 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:25.918643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:25.918707 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:25.918799 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:25.919875 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.920605 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.922805 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.923344 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.923482 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:25.924829 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:25.924834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:25.924837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:25.924853 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:25.924894 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:25.924906 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:25.925000 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:25.925249 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 22:48:25.925347 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":200,"threads_started":1}
I20260809 22:48:25.926606 8748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:25.915997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44537 (local address 127.21.27.129:43039)
0809 22:48:25.916825 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:25.916859 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:25.918421 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:25.918982 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:25.918989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:25.919063 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:25.919723 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:25.919740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.920775 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.920780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:25.922664 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:25.922675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:25.923564 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:25.923570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:25.923792 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:25.924513 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:25.924516 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:25.924562 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:25.924642 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:25.924999 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:25.925012 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:25.925022 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:48:25.925225 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 22:48:25.925579 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":260,"threads_started":1}
I20260809 22:48:25.928037 8744 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44537
I20260809 22:48:25.928342 8744 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:25.929029 8744 heartbeater.cc:511] Master 127.21.27.190:44537 requested a full tablet report, sending...
I20260809 22:48:25.930997 8538 ts_manager.cc:194] Registered new tserver with Master: df064aab18384635afffbcb462f30030 (127.21.27.129:37897)
I20260809 22:48:25.932468 8538 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.129:43039
W20260809 22:48:26.257613 8749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:26.257882 8749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:26.257969 8749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:26.265218 8749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:26.265312 8749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:26.265415 8749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:26.274389 8749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:26.275642 8749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:26.277328 8749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:26.291607 8755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:26.291962 8754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:26.292112 8757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:26.292683 8749 server_base.cc:1034] running on GCE node
I20260809 22:48:26.293304 8749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:26.294140 8749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:26.295369 8749 hybrid_clock.cc:648] HybridClock initialized: now 1786315706295327 us; error 37 us; skew 500 ppm
I20260809 22:48:26.295683 8749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:26.297868 8749 webserver.cc:460] Webserver started at http://127.21.27.130:42135/ using document root <none> and password file <none>
I20260809 22:48:26.298517 8749 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:26.298643 8749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:26.298964 8749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:26.301802 8749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "9bfe189788fc472e8ddaad87b4fc0e8f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "5853cdd94749f799d1698f6c6aa23694"
server_key_iv: "5b1af8b13e1afd1d301478c36b78dc39"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.302604 8749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "9bfe189788fc472e8ddaad87b4fc0e8f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "5853cdd94749f799d1698f6c6aa23694"
server_key_iv: "5b1af8b13e1afd1d301478c36b78dc39"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.307511 8749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:48:26.310708 8763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.312115 8749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:26.312258 8749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "9bfe189788fc472e8ddaad87b4fc0e8f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "5853cdd94749f799d1698f6c6aa23694"
server_key_iv: "5b1af8b13e1afd1d301478c36b78dc39"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.312444 8749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:26.330591 8749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:26.331432 8749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:26.331707 8749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:26.332628 8749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:26.334031 8749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:26.334120 8749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.334251 8749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:26.334347 8749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.368198 8749 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:37047
I20260809 22:48:26.368237 8886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:37047 every 8 connection(s)
I20260809 22:48:26.369768 8749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:26.375187 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8749
I20260809 22:48:26.375378 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:26.375932 21614 external_mini_cluster.cc:1428] Setting key 7279e7f36d63ddb3fb43a54640881cbe
I20260809 22:48:26.378968 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--builtin_ntp_servers=127.21.27.148:33505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:26.383577 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.372429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:40457 (local address 127.21.27.190:44537)
0809 22:48:26.372593 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.372598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.374025 (+ 1427us) server_negotiation.cc:407] Connection header received
0809 22:48:26.376393 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.376400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.376498 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.376610 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:26.377982 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.379125 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.380730 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.381147 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.381287 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.382700 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:26.382705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:26.382708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:26.382725 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:26.382757 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:26.382769 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:26.382865 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.383095 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.383219 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:48:26.384570 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.372982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44537 (local address 127.21.27.130:40457)
0809 22:48:26.373885 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:26.373918 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:26.376069 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:26.376862 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:26.376870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:26.376950 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:26.377717 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.377744 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.379361 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.379366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.380580 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.380592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.381372 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.381379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.381628 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.382359 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:26.382361 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:26.382399 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:26.382483 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:26.382866 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:26.382879 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:26.382889 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:48:26.383083 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 22:48:26.383551 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260809 22:48:26.386135 8888 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44537
I20260809 22:48:26.386458 8888 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:26.387246 8888 heartbeater.cc:511] Master 127.21.27.190:44537 requested a full tablet report, sending...
I20260809 22:48:26.388643 8538 ts_manager.cc:194] Registered new tserver with Master: 9bfe189788fc472e8ddaad87b4fc0e8f (127.21.27.130:37047)
I20260809 22:48:26.389565 8538 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.130:40457
W20260809 22:48:26.704712 8892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:26.704967 8892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:26.705163 8892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:26.712009 8892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:26.712091 8892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:26.712306 8892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:26.721079 8892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:44537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:26.722276 8892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:26.723919 8892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:26.736811 8903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:26.737035 8898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:26.736809 8899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:26.737643 8892 server_base.cc:1034] running on GCE node
I20260809 22:48:26.738206 8892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:26.739108 8892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:26.740370 8892 hybrid_clock.cc:648] HybridClock initialized: now 1786315706740331 us; error 34 us; skew 500 ppm
I20260809 22:48:26.740684 8892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:26.742952 8892 webserver.cc:460] Webserver started at http://127.21.27.131:38787/ using document root <none> and password file <none>
I20260809 22:48:26.743667 8892 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:26.743786 8892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:26.744107 8892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:26.746853 8892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "bd11d1eb830e4808b547c8818d31c78f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "00a53021b85db1e0c63010be7fa2f39f"
server_key_iv: "a0e1ec58e4706bba18a952e3cb023c3c"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.747700 8892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "bd11d1eb830e4808b547c8818d31c78f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "00a53021b85db1e0c63010be7fa2f39f"
server_key_iv: "a0e1ec58e4706bba18a952e3cb023c3c"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.752621 8892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:48:26.756013 8909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.757205 8892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 22:48:26.757381 8892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "bd11d1eb830e4808b547c8818d31c78f"
format_stamp: "Formatted at 2026-08-09 22:48:26 on dist-test-slave-chsk"
server_key: "00a53021b85db1e0c63010be7fa2f39f"
server_key_iv: "a0e1ec58e4706bba18a952e3cb023c3c"
server_key_version: "encryptionkey@0"
I20260809 22:48:26.757581 8892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:26.787751 8892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:26.788528 8892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:26.788823 8892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:26.789745 8892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:26.791118 8892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:26.791285 8892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.791415 8892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:26.791504 8892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:26.825143 8892 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:40249
I20260809 22:48:26.825243 9037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:40249 every 8 connection(s)
I20260809 22:48:26.826463 8892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:26.828154 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 8892
I20260809 22:48:26.828346 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:26.828799 21614 external_mini_cluster.cc:1428] Setting key 2a8f1a0b92779bcaec1a3a945588d9b5
I20260809 22:48:26.840845 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.830704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:55057 (local address 127.21.27.190:44537)
0809 22:48:26.830926 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.830933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.832712 (+ 1779us) server_negotiation.cc:407] Connection header received
0809 22:48:26.834471 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.834476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.834562 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.834687 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:26.835763 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.836503 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.838008 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.838446 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.838598 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.839998 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:26.840004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:26.840009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:26.840036 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:26.840071 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:26.840084 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:26.840180 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.840405 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.840505 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:48:26.841802 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.831393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:44537 (local address 127.21.27.131:55057)
0809 22:48:26.832536 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:26.832589 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:26.834222 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:26.834911 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:26.834919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:26.834999 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:26.835606 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.835623 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.836668 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.836672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.837859 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.837872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.838670 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.838676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.838934 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.839695 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:26.839697 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:26.839735 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:26.839804 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:26.840181 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:26.840194 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:26.840205 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:48:26.840376 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 22:48:26.840742 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":349,"threads_started":1}
I20260809 22:48:26.843348 9038 heartbeater.cc:347] Connected to a master server at 127.21.27.190:44537
I20260809 22:48:26.843715 9038 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:26.844522 9038 heartbeater.cc:511] Master 127.21.27.190:44537 requested a full tablet report, sending...
I20260809 22:48:26.845844 8538 ts_manager.cc:194] Registered new tserver with Master: bd11d1eb830e4808b547c8818d31c78f (127.21.27.131:40249)
I20260809 22:48:26.846662 8538 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.131:55057
I20260809 22:48:26.856376 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:26.866603 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.860671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.21.27.190:44537)
0809 22:48:26.860852 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.860857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.861286 (+ 429us) server_negotiation.cc:407] Connection header received
0809 22:48:26.861465 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.861469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.861564 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.861682 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:26.862247 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.863086 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.864438 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.864844 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.864934 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.865940 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:26.865944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:26.865947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:26.865963 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:26.865994 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:26.866006 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:26.866074 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.866202 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.866272 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:48:26.870199 8538 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:26.877058 8538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:26.899112 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.888984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:37047 (local address 127.0.0.1:39874)
0809 22:48:26.889866 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:26.889923 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:26.890134 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:26.891447 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:26.891456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:26.891498 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:26.892013 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.892025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.893991 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.893995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.896260 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.896272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.898138 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.898145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.898366 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.898417 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:48:26.898510 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:48:26.898702 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":1,"thread_start_us":1820,"threads_started":1}
I20260809 22:48:26.899888 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.889043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39874 (local address 127.21.27.130:37047)
0809 22:48:26.890788 (+ 1745us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.890799 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.890832 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:48:26.890939 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.890946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.891212 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.892049 (+ 837us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:26.892311 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.893847 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.896450 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.897912 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.898949 (+ 1037us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.899257 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.899333 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.899492 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1581,"thread_start_us":1441,"threads_started":1}
I20260809 22:48:26.900756 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.889021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:40249 (local address 127.0.0.1:48064)
0809 22:48:26.891731 (+ 2710us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:26.891754 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:26.891870 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:26.893421 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:26.893426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:26.893450 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:26.893816 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.893827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.895780 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.895786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.897650 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.897699 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.899940 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.899947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.900123 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.900164 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 22:48:26.900266 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 22:48:26.900385 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1954,"spinlock_wait_cycles":31232,"thread_start_us":1002,"threads_started":1}
I20260809 22:48:26.901041 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.889628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.21.27.131:40249)
0809 22:48:26.892643 (+ 3015us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.892652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.892684 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:48:26.892775 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.892783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.892970 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.893144 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:26.894014 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.895639 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.897911 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.899550 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.899679 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.899926 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.900336 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.900512 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2823,"thread_start_us":2642,"threads_started":1}
I20260809 22:48:26.901808 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.892293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.21.27.129:37897)
0809 22:48:26.892908 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:26.892915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:26.893455 (+ 540us) server_negotiation.cc:407] Connection header received
0809 22:48:26.893706 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:26.893713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:26.893910 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:26.894103 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:26.895190 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.896500 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.898898 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.900393 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.900518 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.900734 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:26.901292 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 22:48:26.901451 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":462,"threads_started":1}
I20260809 22:48:26.902303 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:26.891270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:57926)
0809 22:48:26.893334 (+ 2064us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:26.893348 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:26.893491 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:26.894359 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:26.894363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:26.894385 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:26.894986 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.894994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.896641 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.896646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.898710 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:26.898722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:26.900873 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:26.900882 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:26.901073 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:26.901115 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:48:26.901216 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:48:26.901341 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1946,"mutex_wait_us":173,"thread_start_us":1796,"threads_started":1}
I20260809 22:48:26.903867 8814 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 (DEFAULT_TABLE table=test-table [id=eeb51b00de9b41d9a9e1038d0bff613f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:26.905028 8814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba7582e93b3476996ffebfa7567eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:26.904724 8965 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 (DEFAULT_TABLE table=test-table [id=eeb51b00de9b41d9a9e1038d0bff613f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:26.906083 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba7582e93b3476996ffebfa7567eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:26.905974 8673 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 (DEFAULT_TABLE table=test-table [id=eeb51b00de9b41d9a9e1038d0bff613f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:26.907624 8673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba7582e93b3476996ffebfa7567eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:26.923372 9060 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Bootstrap starting.
I20260809 22:48:26.926095 9061 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Bootstrap starting.
I20260809 22:48:26.928508 9059 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Bootstrap starting.
I20260809 22:48:26.930008 9061 tablet_bootstrap.cc:654] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:26.931030 9061 log.cc:824] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:26.931695 9059 tablet_bootstrap.cc:654] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:26.932448 9060 tablet_bootstrap.cc:654] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:26.932950 9059 log.cc:824] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:26.933701 9060 log.cc:824] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:26.935249 8744 heartbeater.cc:503] Master 127.21.27.190:44537 was elected leader, sending a full tablet report...
I20260809 22:48:26.936060 9061 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: No bootstrap required, opened a new log
I20260809 22:48:26.936407 9061 ts_tablet_manager.cc:1397] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260809 22:48:26.942175 9059 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: No bootstrap required, opened a new log
I20260809 22:48:26.942466 9059 ts_tablet_manager.cc:1397] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260809 22:48:26.943454 9060 tablet_bootstrap.cc:492] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: No bootstrap required, opened a new log
I20260809 22:48:26.943771 9060 ts_tablet_manager.cc:1397] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.013s
I20260809 22:48:26.948050 9061 raft_consensus.cc:348] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.948503 9061 raft_consensus.cc:374] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:26.948652 9061 raft_consensus.cc:729] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df064aab18384635afffbcb462f30030, State: Initialized, Role: FOLLOWER
I20260809 22:48:26.949302 9061 consensus_queue.cc:260] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.951100 9061 ts_tablet_manager.cc:1428] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260809 22:48:26.951548 9060 raft_consensus.cc:348] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.951987 9060 raft_consensus.cc:374] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:26.952101 9060 raft_consensus.cc:729] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd11d1eb830e4808b547c8818d31c78f, State: Initialized, Role: FOLLOWER
I20260809 22:48:26.952709 9060 consensus_queue.cc:260] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.952706 9059 raft_consensus.cc:348] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.953126 9059 raft_consensus.cc:374] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:26.953308 9059 raft_consensus.cc:729] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bfe189788fc472e8ddaad87b4fc0e8f, State: Initialized, Role: FOLLOWER
I20260809 22:48:26.953769 9038 heartbeater.cc:503] Master 127.21.27.190:44537 was elected leader, sending a full tablet report...
I20260809 22:48:26.954425 9060 ts_tablet_manager.cc:1428] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Time spent starting tablet: real 0.011s user 0.006s sys 0.003s
I20260809 22:48:26.954257 9059 consensus_queue.cc:260] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:26.955404 8888 heartbeater.cc:503] Master 127.21.27.190:44537 was elected leader, sending a full tablet report...
I20260809 22:48:26.956072 9059 ts_tablet_manager.cc:1428] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 22:48:27.061463 9066 raft_consensus.cc:482] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:27.061838 9066 raft_consensus.cc:504] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:27.063680 9066 leader_election.cc:288] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bfe189788fc472e8ddaad87b4fc0e8f (127.21.27.130:37047), df064aab18384635afffbcb462f30030 (127.21.27.129:37897)
I20260809 22:48:27.069389 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.063962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:37047 (local address 127.21.27.131:47767)
0809 22:48:27.064144 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.064160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:27.064293 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:27.064939 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:27.064944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:27.064978 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:27.065159 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:27.065168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.066235 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.066240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:27.067587 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:27.067612 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.068655 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.068660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:27.068833 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.068857 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:48:27.068958 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:48:27.069092 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 22:48:27.069379 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.064282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:47767 (local address 127.21.27.130:37047)
0809 22:48:27.064502 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.064510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.064533 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:48:27.064639 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.064647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.064747 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.064846 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:27.065350 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.066051 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.067767 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.068502 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.068663 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.068734 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.068997 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.069126 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:48:27.070210 9068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.064343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.21.27.131:41003)
0809 22:48:27.064971 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.064989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:27.065150 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:27.065489 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:27.065493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:27.065515 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:27.065844 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:27.065853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.066912 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.066923 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:27.068322 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:27.068333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.069456 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.069462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:27.069666 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.069705 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:48:27.069797 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 22:48:27.069916 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":1,"thread_start_us":332,"threads_started":1}
I20260809 22:48:27.070225 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.064651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:41003 (local address 127.21.27.129:37897)
0809 22:48:27.064860 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.064867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.065072 (+ 205us) server_negotiation.cc:407] Connection header received
0809 22:48:27.065290 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.065295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.065374 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.065473 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:27.065987 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.066765 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.068502 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.069279 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.069444 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.069509 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.069853 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.069952 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:48:27.070571 8837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba7582e93b3476996ffebfa7567eba8" candidate_uuid: "bd11d1eb830e4808b547c8818d31c78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" is_pre_election: true
I20260809 22:48:27.071079 8837 raft_consensus.cc:2454] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11d1eb830e4808b547c8818d31c78f in term 0.
I20260809 22:48:27.071373 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba7582e93b3476996ffebfa7567eba8" candidate_uuid: "bd11d1eb830e4808b547c8818d31c78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df064aab18384635afffbcb462f30030" is_pre_election: true
I20260809 22:48:27.071857 8911 leader_election.cc:302] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bfe189788fc472e8ddaad87b4fc0e8f, bd11d1eb830e4808b547c8818d31c78f; no voters:
I20260809 22:48:27.071949 8696 raft_consensus.cc:2454] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd11d1eb830e4808b547c8818d31c78f in term 0.
I20260809 22:48:27.072296 9066 raft_consensus.cc:2781] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:27.072481 9066 raft_consensus.cc:482] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:27.072619 9066 raft_consensus.cc:3037] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:27.074833 9066 raft_consensus.cc:504] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:27.075526 9066 leader_election.cc:288] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [CANDIDATE]: Term 1 election: Requested vote from peers 9bfe189788fc472e8ddaad87b4fc0e8f (127.21.27.130:37047), df064aab18384635afffbcb462f30030 (127.21.27.129:37897)
I20260809 22:48:27.075984 8837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba7582e93b3476996ffebfa7567eba8" candidate_uuid: "bd11d1eb830e4808b547c8818d31c78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f"
I20260809 22:48:27.076128 8696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba7582e93b3476996ffebfa7567eba8" candidate_uuid: "bd11d1eb830e4808b547c8818d31c78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df064aab18384635afffbcb462f30030"
I20260809 22:48:27.076179 8837 raft_consensus.cc:3037] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:27.076367 8696 raft_consensus.cc:3037] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 0 FOLLOWER]: Advancing to term 1
W20260809 22:48:27.078244 9039 tablet.cc:2369] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:27.078755 8696 raft_consensus.cc:2454] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11d1eb830e4808b547c8818d31c78f in term 1.
I20260809 22:48:27.078783 8837 raft_consensus.cc:2454] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd11d1eb830e4808b547c8818d31c78f in term 1.
I20260809 22:48:27.079193 8911 leader_election.cc:302] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd11d1eb830e4808b547c8818d31c78f, df064aab18384635afffbcb462f30030; no voters:
I20260809 22:48:27.079505 9066 raft_consensus.cc:2781] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:27.079952 9066 raft_consensus.cc:686] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 1 LEADER]: Becoming Leader. State: Replica: bd11d1eb830e4808b547c8818d31c78f, State: Running, Role: LEADER
I20260809 22:48:27.080403 9066 consensus_queue.cc:237] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } }
I20260809 22:48:27.085208 8538 catalog_manager.cc:5495] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f reported cstate change: term changed from 0 to 1, leader changed from <none> to bd11d1eb830e4808b547c8818d31c78f (127.21.27.131). New cstate: current_term: 1 leader_uuid: "bd11d1eb830e4808b547c8818d31c78f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd11d1eb830e4808b547c8818d31c78f" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 } health_report { overall_health: UNKNOWN } } }
W20260809 22:48:27.121825 8890 tablet.cc:2369] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:27.142232 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.136964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.21.27.131:40249)
0809 22:48:27.137183 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.137188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.137207 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:48:27.137382 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.137386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.137465 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.137594 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:27.137985 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.138722 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.140092 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.140499 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.140599 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.141057 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:27.141580 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:27.141692 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.141802 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.141867 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:48:27.150486 8696 raft_consensus.cc:1261] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 1 FOLLOWER]: Refusing update from remote peer bd11d1eb830e4808b547c8818d31c78f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:27.150514 8837 raft_consensus.cc:1261] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 1 FOLLOWER]: Refusing update from remote peer bd11d1eb830e4808b547c8818d31c78f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:27.151607 9069 consensus_queue.cc:1035] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [LEADER]: Connected to new peer: Peer: permanent_uuid: "df064aab18384635afffbcb462f30030" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:27.152004 9066 consensus_queue.cc:1035] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bfe189788fc472e8ddaad87b4fc0e8f" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 37047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:27.162573 9073 mvcc.cc:205] Tried to move back new op lower bound from 7316749136478343168 to 7316749136205111296. Current Snapshot: MvccSnapshot[applied={T|T < 7316749136478343168}]
I20260809 22:48:27.163718 9074 mvcc.cc:205] Tried to move back new op lower bound from 7316749136478343168 to 7316749136205111296. Current Snapshot: MvccSnapshot[applied={T|T < 7316749136478343168}]
W20260809 22:48:27.165516 8745 tablet.cc:2369] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:27.177162 8538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:27.177462 8538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:27.181131 8538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6714b3059bb14d84aab501222dbef3ee: Sending DeleteTablet for 3 replicas of tablet 1ba7582e93b3476996ffebfa7567eba8
I20260809 22:48:27.182432 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:27 UTC) from {username='slave'} at 127.0.0.1:48064
I20260809 22:48:27.182821 8814 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:27 UTC) from {username='slave'} at 127.0.0.1:39874
I20260809 22:48:27.183423 8673 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba7582e93b3476996ffebfa7567eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:27 UTC) from {username='slave'} at 127.0.0.1:57926
I20260809 22:48:27.183676 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8604
I20260809 22:48:27.200742 9085 tablet_replica.cc:330] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: stopping tablet replica
I20260809 22:48:27.202837 9086 tablet_replica.cc:330] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: stopping tablet replica
I20260809 22:48:27.202850 9085 raft_consensus.cc:2229] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:27.203382 9086 raft_consensus.cc:2229] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:27.204027 9086 raft_consensus.cc:2258] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:27.204697 9084 tablet_replica.cc:330] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: stopping tablet replica
I20260809 22:48:27.204865 9085 raft_consensus.cc:2258] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:27.207309 9086 ts_tablet_manager.cc:1905] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:27.207319 9085 ts_tablet_manager.cc:1905] T 1ba7582e93b3476996ffebfa7567eba8 P df064aab18384635afffbcb462f30030: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:27.208776 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.185677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.21.27.129:37897)
0809 22:48:27.185914 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.185924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.196563 (+ 10639us) server_negotiation.cc:407] Connection header received
0809 22:48:27.197160 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.197170 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.197393 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.197495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:27.198354 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.199981 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.204566 (+ 4585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.205259 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.205412 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.206983 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:27.206995 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:27.207013 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:27.207826 (+ 813us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:27.207898 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:27.207924 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:27.208012 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.208273 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.208392 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:48:27.210225 8739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:27.212810 9084 raft_consensus.cc:2229] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:27.213755 9084 raft_consensus.cc:2258] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:27.214396 9086 ts_tablet_manager.cc:1918] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:27.214597 9086 log.cc:1196] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/1ba7582e93b3476996ffebfa7567eba8
I20260809 22:48:27.215204 9086 ts_tablet_manager.cc:1939] T 1ba7582e93b3476996ffebfa7567eba8 P 9bfe189788fc472e8ddaad87b4fc0e8f: Deleting consensus metadata
I20260809 22:48:27.216923 9084 ts_tablet_manager.cc:1905] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:27.217358 8522 catalog_manager.cc:4841] TS 9bfe189788fc472e8ddaad87b4fc0e8f (127.21.27.130:37047): tablet 1ba7582e93b3476996ffebfa7567eba8 (table test-table [id=eeb51b00de9b41d9a9e1038d0bff613f]) successfully deleted
I20260809 22:48:27.223222 9084 ts_tablet_manager.cc:1918] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:27.223429 9084 log.cc:1196] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/1ba7582e93b3476996ffebfa7567eba8
I20260809 22:48:27.223928 9084 ts_tablet_manager.cc:1939] T 1ba7582e93b3476996ffebfa7567eba8 P bd11d1eb830e4808b547c8818d31c78f: Deleting consensus metadata
I20260809 22:48:27.225507 8524 catalog_manager.cc:4841] TS bd11d1eb830e4808b547c8818d31c78f (127.21.27.131:40249): tablet 1ba7582e93b3476996ffebfa7567eba8 (table test-table [id=eeb51b00de9b41d9a9e1038d0bff613f]) successfully deleted
I20260809 22:48:27.367815 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8604
I20260809 22:48:27.394095 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8749
W20260809 22:48:27.394162 8522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.27.129:37897 (error 108)
I20260809 22:48:27.407501 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.396032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.21.27.130:37047)
0809 22:48:27.396290 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.396299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.396324 (+ 25us) server_negotiation.cc:407] Connection header received
0809 22:48:27.396404 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.396412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.396557 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.396698 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:27.397532 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.399158 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.401874 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.402884 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.403044 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.404957 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:27.404971 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:27.404993 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:27.405822 (+ 829us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:27.405887 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:27.405914 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:27.406032 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.406760 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.406922 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:48:27.408390 8882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:27.408771 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.405933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60872)
0809 22:48:27.408326 (+ 2393us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.408557 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2239,"thread_start_us":2151,"threads_started":1}
W20260809 22:48:27.409162 8522 catalog_manager.cc:4584] TS df064aab18384635afffbcb462f30030 (127.21.27.129:37897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ba7582e93b3476996ffebfa7567eba8: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
I20260809 22:48:27.463274 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.462691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60886)
0809 22:48:27.462945 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.463061 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 22:48:27.464552 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.464087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60894)
0809 22:48:27.464279 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.464377 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 22:48:27.500905 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8749
I20260809 22:48:27.520725 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8892
I20260809 22:48:27.524724 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.524288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60900)
0809 22:48:27.524482 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.524572 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 22:48:27.525889 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.525478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60908)
0809 22:48:27.525630 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.525726 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:48:27.529489 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.521721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.21.27.131:40249)
0809 22:48:27.521897 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:27.521902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:27.521921 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:48:27.522075 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:27.522079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:27.522187 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:27.522275 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:27.522813 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.523724 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.525703 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:27.526343 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:27.526484 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:27.527983 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:27.527993 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:27.528008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:27.528649 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:27.528696 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:27.528716 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:27.528815 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:27.529011 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 22:48:27.529095 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:48:27.530478 9033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:27.607306 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.606686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60916)
0809 22:48:27.606920 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.607032 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 22:48:27.608440 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.608027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60924)
0809 22:48:27.608236 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.608305 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260809 22:48:27.653438 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8892
I20260809 22:48:27.670967 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 8503
I20260809 22:48:27.672372 8572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:27.786264 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:27.785265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37897 (local address 127.0.0.1:60930)
0809 22:48:27.785696 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:27.785925 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.129:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:48:27.786456 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 8503
2026-08-09T22:48:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2945 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:27 dist-test-slave-chsk krb5kdc[9097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:27 dist-test-slave-chsk krb5kdc[9097](info): set up 2 sockets
Aug 09 22:48:27 dist-test-slave-chsk krb5kdc[9097](info): commencing operation
krb5kdc: starting...
W20260809 22:48:28.959337 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:29 dist-test-slave-chsk krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:48:29.255939 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45761
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:29.593115 9110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:29.593425 9110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:29.593586 9110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:29.600925 9110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:29.601060 9110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:29.601145 9110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:29.601244 9110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:29.601328 9110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:29.611169 9110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:29.612576 9110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:29.613855 9110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:29.625903 9116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:29.625952 9118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:29.625981 9115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:29.627461 9110 server_base.cc:1034] running on GCE node
I20260809 22:48:29.628363 9110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:29.629906 9110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:29.631189 9110 hybrid_clock.cc:648] HybridClock initialized: now 1786315709631079 us; error 50 us; skew 500 ppm
I20260809 22:48:29.631606 9110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:29 dist-test-slave-chsk krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:29.634840 9110 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:48:29.636461 9110 webserver.cc:460] Webserver started at http://127.21.27.190:41667/ using document root <none> and password file <none>
I20260809 22:48:29.637136 9110 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:29.637270 9110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:29.637609 9110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:29.640733 9110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "70d1f7d59ba34c2e9b1abdc744d1619c"
format_stamp: "Formatted at 2026-08-09 22:48:29 on dist-test-slave-chsk"
server_key: "0b8f58f1757d88e81561e1c07f454a54"
server_key_iv: "2fd7e27c6bf82c5627e76d4961ada45c"
server_key_version: "encryptionkey@0"
I20260809 22:48:29.641543 9110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "70d1f7d59ba34c2e9b1abdc744d1619c"
format_stamp: "Formatted at 2026-08-09 22:48:29 on dist-test-slave-chsk"
server_key: "0b8f58f1757d88e81561e1c07f454a54"
server_key_iv: "2fd7e27c6bf82c5627e76d4961ada45c"
server_key_version: "encryptionkey@0"
I20260809 22:48:29.646699 9110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 22:48:29.650005 9127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:29.651284 9110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:29.651473 9110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "70d1f7d59ba34c2e9b1abdc744d1619c"
format_stamp: "Formatted at 2026-08-09 22:48:29 on dist-test-slave-chsk"
server_key: "0b8f58f1757d88e81561e1c07f454a54"
server_key_iv: "2fd7e27c6bf82c5627e76d4961ada45c"
server_key_version: "encryptionkey@0"
I20260809 22:48:29.651707 9110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:29.670799 9110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:29.675207 9110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:29.675518 9110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:29.694687 9110 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45761
I20260809 22:48:29.694865 9183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45761 every 8 connection(s)
I20260809 22:48:29.696161 9110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:29.699934 9184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:29.703938 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9110
I20260809 22:48:29.704152 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:29.704629 21614 external_mini_cluster.cc:1428] Setting key 21a572db5f57a2c23f4bcbea556f607e
I20260809 22:48:29.711351 9184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: Bootstrap starting.
I20260809 22:48:29.715011 9184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:29.716284 9184 log.cc:824] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:29.719802 9184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: No bootstrap required, opened a new log
Aug 09 22:48:29 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:29.727684 9184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } }
I20260809 22:48:29.728073 9184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:29.728195 9184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d1f7d59ba34c2e9b1abdc744d1619c, State: Initialized, Role: FOLLOWER
I20260809 22:48:29.728881 9184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } }
I20260809 22:48:29.729236 9184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:29.729419 9184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:29.729635 9184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:29.729808 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:29.706581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57850 (local address 127.21.27.190:45761)
0809 22:48:29.707452 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:29.707464 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:29.707504 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:48:29.709856 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:29.709878 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:29.710239 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:29.710546 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:29.712648 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:29.713950 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:29.715996 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:29.716747 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:29.716975 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:29.721786 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:29.721793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:29.721810 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:29.721852 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:29.725131 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:29.725871 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:29.725877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:29.725882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:29.726063 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:29.726472 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:29.726476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:29.726478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:29.726984 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:29.727312 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:29.727772 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:48:29.728242 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":371,"threads_started":1}
I20260809 22:48:29.733561 9184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } }
I20260809 22:48:29.734205 9184 leader_election.cc:302] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70d1f7d59ba34c2e9b1abdc744d1619c; no voters:
I20260809 22:48:29.735787 9184 leader_election.cc:288] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:29.735889 9189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:29.738278 9184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:29.739507 9189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [term 1 LEADER]: Becoming Leader. State: Replica: 70d1f7d59ba34c2e9b1abdc744d1619c, State: Running, Role: LEADER
I20260809 22:48:29.740020 9189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } }
I20260809 22:48:29.742815 9190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } } }
I20260809 22:48:29.743373 9190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:29.747953 9191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70d1f7d59ba34c2e9b1abdc744d1619c. Latest consensus state: current_term: 1 leader_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70d1f7d59ba34c2e9b1abdc744d1619c" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45761 } } }
I20260809 22:48:29.748241 9191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:29.749678 9196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:29.755044 9196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:29.764060 9196 catalog_manager.cc:1347] Generated new cluster ID: 058cf0206f92491aaa5eb84c0c6e30ae
I20260809 22:48:29.764186 9196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:29.786937 9196 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:29.788364 9196 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:29.797111 9196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: Generated new TSK 0
I20260809 22:48:29.797894 9196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:48:29.882232 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:30.216257 9213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:30.216507 9213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:30.216719 9213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:30.223841 9213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:30.223938 9213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:30.224098 9213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:30.233964 9213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:30.235239 9213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:30.236563 9213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:30.250358 9218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:30.250485 9219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:30.250491 9221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:30.251361 9213 server_base.cc:1034] running on GCE node
I20260809 22:48:30.251881 9213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:30.252720 9213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:30.253911 9213 hybrid_clock.cc:648] HybridClock initialized: now 1786315710253859 us; error 50 us; skew 500 ppm
I20260809 22:48:30.254165 9213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:30 dist-test-slave-chsk krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:30.257473 9213 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:48:30.258818 9213 webserver.cc:460] Webserver started at http://127.21.27.129:41395/ using document root <none> and password file <none>
I20260809 22:48:30.259536 9213 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:30.259683 9213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:30.260001 9213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:30.262823 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "dce588b146ee45639d72c1abf654cbf4"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "d50de0e6714b53b7f5cc290216f8d1f0"
server_key_iv: "7c20f0e73d3bf50e8fc5feb7de3b3ad0"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.263602 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "dce588b146ee45639d72c1abf654cbf4"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "d50de0e6714b53b7f5cc290216f8d1f0"
server_key_iv: "7c20f0e73d3bf50e8fc5feb7de3b3ad0"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.268389 9213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 22:48:30.271622 9230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.272888 9213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:48:30.273073 9213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "dce588b146ee45639d72c1abf654cbf4"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "d50de0e6714b53b7f5cc290216f8d1f0"
server_key_iv: "7c20f0e73d3bf50e8fc5feb7de3b3ad0"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.273272 9213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:30.297442 9213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:30.302716 9213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:30.302973 9213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:30.304009 9213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:30.305290 9213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:30.305364 9213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.305446 9213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:30.305491 9213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.339516 9213 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:33641
I20260809 22:48:30.339597 9358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:33641 every 8 connection(s)
I20260809 22:48:30.340786 9213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:30.351003 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9213
I20260809 22:48:30.351246 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:30.351725 21614 external_mini_cluster.cc:1428] Setting key ff27cacc5b61799ddfe603283cd2fbda
Aug 09 22:48:30 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:30.364277 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:30.343342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:39709 (local address 127.21.27.190:45761)
0809 22:48:30.343804 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:30.343808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:30.344537 (+ 729us) server_negotiation.cc:407] Connection header received
0809 22:48:30.346363 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:30.346368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:30.346459 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:30.346584 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:30.348161 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.348893 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.350369 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.350961 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.351113 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:30.358265 (+ 7152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:30.358270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:30.358274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:30.358304 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:30.360992 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:30.361914 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:30.361918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:30.361920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:30.362048 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:30.362608 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:30.362611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:30.362613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:30.362862 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:30.362978 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:30.363867 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 22:48:30.364030 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":251,"threads_started":1}
I20260809 22:48:30.365751 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:30.343716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45761 (local address 127.21.27.129:39709)
0809 22:48:30.344386 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:30.344422 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:30.346104 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:30.346831 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:30.346838 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:30.347408 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:30.347984 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:30.348000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.349055 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.349060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:30.350224 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:30.350237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.351255 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.351262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:30.351506 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:30.355578 (+ 4072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:30.355600 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:30.358032 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:30.361137 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:30.361147 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:30.361163 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:30.361796 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:30.362153 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:30.362161 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:30.362167 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:30.362416 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:30.363001 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:30.363011 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:30.363458 (+ 447us) client_negotiation.cc:769] Sending connection context
0809 22:48:30.363751 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 22:48:30.364296 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":265,"threads_started":1}
I20260809 22:48:30.367226 9359 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45761
I20260809 22:48:30.367517 9359 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:30.368191 9359 heartbeater.cc:511] Master 127.21.27.190:45761 requested a full tablet report, sending...
I20260809 22:48:30.370252 9146 ts_manager.cc:194] Registered new tserver with Master: dce588b146ee45639d72c1abf654cbf4 (127.21.27.129:33641)
I20260809 22:48:30.371760 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:39709
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:48:30.416664 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:30.747100 9366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:30.747499 9366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:30.747684 9366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:30.755072 9366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:30.755208 9366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:30.755395 9366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:30.765587 9366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:30.766825 9366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:30.768193 9366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:30.781702 9372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:30.781726 9374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:30.781737 9371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:30.782619 9366 server_base.cc:1034] running on GCE node
I20260809 22:48:30.783619 9366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:30.784653 9366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:30.785897 9366 hybrid_clock.cc:648] HybridClock initialized: now 1786315710785834 us; error 57 us; skew 500 ppm
I20260809 22:48:30.786232 9366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:30 dist-test-slave-chsk krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:30.794239 9366 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:48:30.795936 9366 webserver.cc:460] Webserver started at http://127.21.27.130:37365/ using document root <none> and password file <none>
I20260809 22:48:30.796648 9366 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:30.796797 9366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:30.797130 9366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:30.800129 9366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "84e18855ce2e4159aeebee0cc04f228b"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "c3afccca0f48d8ace6c00660bfc7c416"
server_key_iv: "f421b360efaf23e0f79710e5ee830684"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.800896 9366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "84e18855ce2e4159aeebee0cc04f228b"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "c3afccca0f48d8ace6c00660bfc7c416"
server_key_iv: "f421b360efaf23e0f79710e5ee830684"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.806442 9366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:48:30.810315 9382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.811820 9366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:30.812004 9366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "84e18855ce2e4159aeebee0cc04f228b"
format_stamp: "Formatted at 2026-08-09 22:48:30 on dist-test-slave-chsk"
server_key: "c3afccca0f48d8ace6c00660bfc7c416"
server_key_iv: "f421b360efaf23e0f79710e5ee830684"
server_key_version: "encryptionkey@0"
I20260809 22:48:30.812230 9366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:30.843021 9366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:30.847752 9366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:30.848120 9366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:30.849190 9366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:30.850721 9366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:30.850837 9366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.851023 9366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:30.851125 9366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:30.886957 9366 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:46141
I20260809 22:48:30.886996 9507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:46141 every 8 connection(s)
I20260809 22:48:30.888315 9366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:30.897070 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9366
I20260809 22:48:30.897261 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:30.897742 21614 external_mini_cluster.cc:1428] Setting key e985e6e02562f286ccea2c4a95edee3c
Aug 09 22:48:30 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:30.908766 9510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:30.890944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:51101 (local address 127.21.27.190:45761)
0809 22:48:30.891493 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:30.891498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:30.892251 (+ 753us) server_negotiation.cc:407] Connection header received
0809 22:48:30.894073 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:30.894078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:30.894168 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:30.894286 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:30.895886 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.896623 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.898343 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.898974 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.899115 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:30.902755 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:30.902760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:30.902764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:30.902796 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:30.905543 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:30.906309 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:30.906314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:30.906316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:30.906442 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:30.906911 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:30.906915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:30.906917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:30.907252 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:30.907424 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:30.908121 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 22:48:30.908351 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":337,"threads_started":1}
I20260809 22:48:30.909765 9511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:30.891422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45761 (local address 127.21.27.130:51101)
0809 22:48:30.892086 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:30.892136 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:30.893811 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:30.894544 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:30.894556 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:30.895072 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:30.895711 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:30.895729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.896807 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.896811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:30.898198 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:30.898211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:30.899230 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:30.899239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:30.899531 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:30.900251 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:30.900274 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:30.902517 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:30.905710 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:30.905719 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:30.905735 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:30.906167 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:30.906556 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:30.906561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:30.906565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:30.906766 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:30.907393 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:30.907402 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:30.907752 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 22:48:30.908018 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 22:48:30.908590 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":280,"threads_started":1}
I20260809 22:48:30.911367 9508 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45761
I20260809 22:48:30.911688 9508 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:30.912395 9508 heartbeater.cc:511] Master 127.21.27.190:45761 requested a full tablet report, sending...
I20260809 22:48:30.913877 9146 ts_manager.cc:194] Registered new tserver with Master: 84e18855ce2e4159aeebee0cc04f228b (127.21.27.130:46141)
I20260809 22:48:30.914711 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:51101
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:48:30.971092 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:31.310667 9515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:31.310946 9515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:31.311170 9515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:31.318781 9515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:31.318943 9515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:31.319185 9515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:31.329121 9515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:31.330430 9515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:31.331828 9515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:31.345818 9520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:31.345772 9521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:31.345811 9523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:31.346544 9515 server_base.cc:1034] running on GCE node
I20260809 22:48:31.347079 9515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:31.348111 9515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:31.349323 9515 hybrid_clock.cc:648] HybridClock initialized: now 1786315711349275 us; error 38 us; skew 500 ppm
I20260809 22:48:31.349639 9515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:31 dist-test-slave-chsk krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:31.352823 9515 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:48:31.354290 9515 webserver.cc:460] Webserver started at http://127.21.27.131:44359/ using document root <none> and password file <none>
I20260809 22:48:31.355024 9515 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:31.355178 9515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:31.355507 9515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:31.358330 9515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "8e1cb301734445c593b1d192bee7712e"
format_stamp: "Formatted at 2026-08-09 22:48:31 on dist-test-slave-chsk"
server_key: "9c623346208c614f8c00c925b827a985"
server_key_iv: "042f662851d4f14c387f5d6ec1284b1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:31.359086 9515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "8e1cb301734445c593b1d192bee7712e"
format_stamp: "Formatted at 2026-08-09 22:48:31 on dist-test-slave-chsk"
server_key: "9c623346208c614f8c00c925b827a985"
server_key_iv: "042f662851d4f14c387f5d6ec1284b1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:31.364192 9515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:48:31.367631 9530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:31.368878 9515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:48:31.369042 9515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "8e1cb301734445c593b1d192bee7712e"
format_stamp: "Formatted at 2026-08-09 22:48:31 on dist-test-slave-chsk"
server_key: "9c623346208c614f8c00c925b827a985"
server_key_iv: "042f662851d4f14c387f5d6ec1284b1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:31.369243 9515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:31.374581 9359 heartbeater.cc:503] Master 127.21.27.190:45761 was elected leader, sending a full tablet report...
I20260809 22:48:31.397169 9515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:31.401657 9515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:31.401908 9515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:31.402851 9515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:31.404354 9515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:31.404429 9515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:31.404512 9515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:31.404557 9515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:31.438926 9515 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:33765
I20260809 22:48:31.439006 9658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:33765 every 8 connection(s)
I20260809 22:48:31.440323 9515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:31.446535 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9515
I20260809 22:48:31.446745 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:31.447351 21614 external_mini_cluster.cc:1428] Setting key b648196c0aa64b65a62ae30f920d83af
Aug 09 22:48:31 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315711, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:31.460974 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.442912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:37333 (local address 127.21.27.190:45761)
0809 22:48:31.443425 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.443430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.444350 (+ 920us) server_negotiation.cc:407] Connection header received
0809 22:48:31.446372 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.446379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.446472 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.446630 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:31.448409 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.449257 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.450840 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.451309 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.451447 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.454956 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:31.454962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:31.454965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:31.455002 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:31.457875 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.458651 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.458657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.458659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.458786 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.459256 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.459260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.459262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.459552 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:31.459683 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.460444 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.460631 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":277,"threads_started":1}
I20260809 22:48:31.462234 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.443467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45761 (local address 127.21.27.131:37333)
0809 22:48:31.444184 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.444229 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.445947 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.446918 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.446926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.447598 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:31.448175 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.448193 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.449435 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.449439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.450681 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.450699 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.451544 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.451551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.451795 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.452434 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:31.452457 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:31.454720 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:31.458038 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:31.458045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:31.458059 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:31.458520 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:31.458891 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:31.458895 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:31.458897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:31.459085 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:31.459688 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:31.459694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:31.460019 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.460348 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.460925 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":292,"threads_started":1}
I20260809 22:48:31.463820 9659 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45761
I20260809 22:48:31.464148 9659 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:31.464956 9659 heartbeater.cc:511] Master 127.21.27.190:45761 requested a full tablet report, sending...
I20260809 22:48:31.466360 9146 ts_manager.cc:194] Registered new tserver with Master: 8e1cb301734445c593b1d192bee7712e (127.21.27.131:33765)
I20260809 22:48:31.467228 9146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:37333
I20260809 22:48:31.475127 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:31.492648 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.479386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.21.27.190:45761)
0809 22:48:31.479613 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.479621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.479983 (+ 362us) server_negotiation.cc:407] Connection header received
0809 22:48:31.480121 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.480125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.480201 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.480281 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:31.481676 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.482428 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.483953 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.484382 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.484483 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.487262 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:31.487267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:31.487271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:31.487303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:31.489800 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.490670 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.490674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.490676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.490793 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.491250 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.491253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.491255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.491554 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:31.491753 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.492111 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.492256 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:48:31.496759 9146 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:31.503983 9146 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:31.529444 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.517448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.21.27.131:33765)
0809 22:48:31.518227 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.518237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.518271 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:48:31.518373 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.518379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.518549 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.518738 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:31.519589 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.521515 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.525235 (+ 3720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.527278 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.527414 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.527741 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.528688 (+ 947us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.528939 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":411,"threads_started":1}
I20260809 22:48:31.530921 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.517086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:33765 (local address 127.0.0.1:48470)
0809 22:48:31.517865 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.517920 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.518104 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.518749 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.518755 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.518787 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:31.519308 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.519329 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.521776 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.521784 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.524992 (+ 3208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.525015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.527989 (+ 2974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.527998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.528303 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.528384 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.530079 (+ 1695us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.530396 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":1059,"threads_started":1}
I20260809 22:48:31.535970 9590 tablet_service.cc:1467] Processing CreateTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 (DEFAULT_TABLE table=test-table [id=9a1026ad349d4a97986daba62814c881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:31.537690 9590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f02b3c8e12804a65b5fcd4078d4cb679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:31.540560 9672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.519581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:46141 (local address 127.0.0.1:54116)
0809 22:48:31.531501 (+ 11920us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.531527 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.531678 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.533124 (+ 1446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.533128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.533156 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:31.533332 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.533343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.535388 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.535393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.537808 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.537827 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.539706 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.539712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.539904 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.539943 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.540052 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.540222 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11746,"thread_start_us":11574,"threads_started":1}
I20260809 22:48:31.541509 9675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.518876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:33641 (local address 127.0.0.1:44898)
0809 22:48:31.519548 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.519571 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.519674 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.534167 (+ 14493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.534173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.534200 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:31.534625 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.534638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.536915 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.536920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.538750 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.538767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.540688 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.540697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.540873 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.540921 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.540995 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.541148 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":352,"threads_started":1}
I20260809 22:48:31.541960 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.520181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.21.27.130:46141)
0809 22:48:31.532263 (+ 12082us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.532272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.532306 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:48:31.532393 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.532399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.532601 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.532833 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:31.533507 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.535130 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.538015 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.539501 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.539666 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.539884 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.541339 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.541638 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11938,"thread_start_us":11786,"threads_started":1}
I20260809 22:48:31.542558 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.519283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44898 (local address 127.21.27.129:33641)
0809 22:48:31.533510 (+ 14227us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.533520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.533548 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:48:31.533670 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.533677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.533936 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.534840 (+ 904us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:31.535045 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.536709 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.538960 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.540418 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.541876 (+ 1458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.542108 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.542190 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.542329 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14038,"thread_start_us":13816,"threads_started":1}
I20260809 22:48:31.545396 9439 tablet_service.cc:1467] Processing CreateTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 (DEFAULT_TABLE table=test-table [id=9a1026ad349d4a97986daba62814c881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:31.546025 9285 tablet_service.cc:1467] Processing CreateTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 (DEFAULT_TABLE table=test-table [id=9a1026ad349d4a97986daba62814c881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:31.546687 9439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f02b3c8e12804a65b5fcd4078d4cb679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:31.547494 9285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f02b3c8e12804a65b5fcd4078d4cb679. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:31.553097 9679 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Bootstrap starting.
I20260809 22:48:31.557040 9679 tablet_bootstrap.cc:654] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:31.558444 9679 log.cc:824] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:31.561591 9683 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Bootstrap starting.
I20260809 22:48:31.561764 9679 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: No bootstrap required, opened a new log
I20260809 22:48:31.562047 9679 ts_tablet_manager.cc:1397] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:48:31.563331 9682 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Bootstrap starting.
I20260809 22:48:31.565389 9683 tablet_bootstrap.cc:654] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:31.565992 9682 tablet_bootstrap.cc:654] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:31.566694 9683 log.cc:824] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:31.566977 9682 log.cc:824] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:31.570127 9683 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: No bootstrap required, opened a new log
I20260809 22:48:31.570410 9683 ts_tablet_manager.cc:1397] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 22:48:31.572618 9682 tablet_bootstrap.cc:492] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: No bootstrap required, opened a new log
I20260809 22:48:31.572960 9682 ts_tablet_manager.cc:1397] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260809 22:48:31.572836 9679 raft_consensus.cc:348] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.573288 9679 raft_consensus.cc:374] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:31.573393 9679 raft_consensus.cc:729] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e1cb301734445c593b1d192bee7712e, State: Initialized, Role: FOLLOWER
I20260809 22:48:31.574004 9679 consensus_queue.cc:260] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.575170 9659 heartbeater.cc:503] Master 127.21.27.190:45761 was elected leader, sending a full tablet report...
I20260809 22:48:31.575814 9679 ts_tablet_manager.cc:1428] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260809 22:48:31.581646 9683 raft_consensus.cc:348] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.582170 9683 raft_consensus.cc:374] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:31.582295 9683 raft_consensus.cc:729] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce588b146ee45639d72c1abf654cbf4, State: Initialized, Role: FOLLOWER
I20260809 22:48:31.582387 9682 raft_consensus.cc:348] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.582827 9682 raft_consensus.cc:374] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:31.582935 9682 raft_consensus.cc:729] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e18855ce2e4159aeebee0cc04f228b, State: Initialized, Role: FOLLOWER
I20260809 22:48:31.582957 9683 consensus_queue.cc:260] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.583571 9682 consensus_queue.cc:260] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.584667 9508 heartbeater.cc:503] Master 127.21.27.190:45761 was elected leader, sending a full tablet report...
I20260809 22:48:31.584887 9683 ts_tablet_manager.cc:1428] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260809 22:48:31.585287 9682 ts_tablet_manager.cc:1428] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
W20260809 22:48:31.593231 9360 tablet.cc:2369] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:31.595824 9688 raft_consensus.cc:482] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:31.596057 9688 raft_consensus.cc:504] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.597615 9688 leader_election.cc:288] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e1cb301734445c593b1d192bee7712e (127.21.27.131:33765), dce588b146ee45639d72c1abf654cbf4 (127.21.27.129:33641)
I20260809 22:48:31.604477 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.598011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:50803 (local address 127.21.27.131:33765)
0809 22:48:31.598239 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.598246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.598781 (+ 535us) server_negotiation.cc:407] Connection header received
0809 22:48:31.599231 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.599241 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.599334 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.599399 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:31.600144 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.600883 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.602644 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.603504 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.603692 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.603770 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.604096 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.604218 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:48:31.604569 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.597958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:33765 (local address 127.21.27.130:50803)
0809 22:48:31.598668 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.598689 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.598911 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.599635 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.599640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.599666 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:31.599992 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.600000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.601016 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.601031 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.602487 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.602499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.603720 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.603727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.603903 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.603958 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.604067 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.604206 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":373,"threads_started":1}
I20260809 22:48:31.604862 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.598938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:59971 (local address 127.21.27.129:33641)
0809 22:48:31.599130 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.599136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.599721 (+ 585us) server_negotiation.cc:407] Connection header received
0809 22:48:31.599940 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.599951 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.600008 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.600103 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:31.600636 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.601325 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.603068 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.603941 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.604115 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.604189 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.604486 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.604613 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:48:31.604918 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.598796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:33641 (local address 127.21.27.130:59971)
0809 22:48:31.599612 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:31.599630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:31.599806 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:31.600120 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:31.600124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:31.600156 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:31.600468 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.600476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.601466 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.601474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.602911 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:31.602923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.604155 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.604161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:31.604330 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.604355 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:48:31.604461 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 22:48:31.604586 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":470,"threads_started":1}
I20260809 22:48:31.605577 9611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f02b3c8e12804a65b5fcd4078d4cb679" candidate_uuid: "84e18855ce2e4159aeebee0cc04f228b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1cb301734445c593b1d192bee7712e" is_pre_election: true
I20260809 22:48:31.605995 9308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f02b3c8e12804a65b5fcd4078d4cb679" candidate_uuid: "84e18855ce2e4159aeebee0cc04f228b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce588b146ee45639d72c1abf654cbf4" is_pre_election: true
I20260809 22:48:31.606175 9611 raft_consensus.cc:2454] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e18855ce2e4159aeebee0cc04f228b in term 0.
I20260809 22:48:31.606560 9308 raft_consensus.cc:2454] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e18855ce2e4159aeebee0cc04f228b in term 0.
I20260809 22:48:31.606896 9383 leader_election.cc:302] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e18855ce2e4159aeebee0cc04f228b, 8e1cb301734445c593b1d192bee7712e; no voters:
I20260809 22:48:31.607437 9688 raft_consensus.cc:2781] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:31.607620 9688 raft_consensus.cc:482] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:31.607756 9688 raft_consensus.cc:3037] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:31.610008 9688 raft_consensus.cc:504] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.610622 9688 leader_election.cc:288] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [CANDIDATE]: Term 1 election: Requested vote from peers 8e1cb301734445c593b1d192bee7712e (127.21.27.131:33765), dce588b146ee45639d72c1abf654cbf4 (127.21.27.129:33641)
I20260809 22:48:31.611128 9611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f02b3c8e12804a65b5fcd4078d4cb679" candidate_uuid: "84e18855ce2e4159aeebee0cc04f228b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e1cb301734445c593b1d192bee7712e"
I20260809 22:48:31.611275 9308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f02b3c8e12804a65b5fcd4078d4cb679" candidate_uuid: "84e18855ce2e4159aeebee0cc04f228b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce588b146ee45639d72c1abf654cbf4"
I20260809 22:48:31.611394 9611 raft_consensus.cc:3037] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:31.611538 9308 raft_consensus.cc:3037] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:31.613716 9611 raft_consensus.cc:2454] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e18855ce2e4159aeebee0cc04f228b in term 1.
I20260809 22:48:31.613806 9308 raft_consensus.cc:2454] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e18855ce2e4159aeebee0cc04f228b in term 1.
I20260809 22:48:31.614195 9383 leader_election.cc:302] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84e18855ce2e4159aeebee0cc04f228b, 8e1cb301734445c593b1d192bee7712e; no voters:
I20260809 22:48:31.614523 9688 raft_consensus.cc:2781] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:31.615007 9688 raft_consensus.cc:686] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 1 LEADER]: Becoming Leader. State: Replica: 84e18855ce2e4159aeebee0cc04f228b, State: Running, Role: LEADER
I20260809 22:48:31.615545 9688 consensus_queue.cc:237] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } }
I20260809 22:48:31.620447 9146 catalog_manager.cc:5495] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e18855ce2e4159aeebee0cc04f228b (127.21.27.130). New cstate: current_term: 1 leader_uuid: "84e18855ce2e4159aeebee0cc04f228b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84e18855ce2e4159aeebee0cc04f228b" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 46141 } health_report { overall_health: HEALTHY } } }
W20260809 22:48:31.640511 9509 tablet.cc:2369] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:31.656569 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.650955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.21.27.130:46141)
0809 22:48:31.651211 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.651216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.651235 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:48:31.651320 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.651324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.651414 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.651553 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:31.651905 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.652672 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.654082 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.654528 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.654724 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.655401 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:31.655905 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:31.656104 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.656308 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.656352 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:48:31.665467 9308 raft_consensus.cc:1261] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 1 FOLLOWER]: Refusing update from remote peer 84e18855ce2e4159aeebee0cc04f228b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:31.665467 9611 raft_consensus.cc:1261] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 1 FOLLOWER]: Refusing update from remote peer 84e18855ce2e4159aeebee0cc04f228b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:31.666589 9691 consensus_queue.cc:1035] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce588b146ee45639d72c1abf654cbf4" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 33641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:31.667011 9688 consensus_queue.cc:1035] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e1cb301734445c593b1d192bee7712e" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:31.680406 9696 mvcc.cc:205] Tried to move back new op lower bound from 7316749154969632768 to 7316749154781597696. Current Snapshot: MvccSnapshot[applied={T|T < 7316749154969632768}]
I20260809 22:48:31.693732 9145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:31.694046 9145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:31.697717 9145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70d1f7d59ba34c2e9b1abdc744d1619c: Sending DeleteTablet for 3 replicas of tablet f02b3c8e12804a65b5fcd4078d4cb679
I20260809 22:48:31.699115 9285 tablet_service.cc:1514] Processing DeleteTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:44898
I20260809 22:48:31.699489 9439 tablet_service.cc:1514] Processing DeleteTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:54116
I20260809 22:48:31.700486 9590 tablet_service.cc:1514] Processing DeleteTablet for tablet f02b3c8e12804a65b5fcd4078d4cb679 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:31 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:48470
I20260809 22:48:31.706547 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9213
I20260809 22:48:31.712697 9710 tablet_replica.cc:330] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: stopping tablet replica
I20260809 22:48:31.714557 9709 tablet_replica.cc:330] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: stopping tablet replica
I20260809 22:48:31.714787 9710 raft_consensus.cc:2229] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:31.715219 9709 raft_consensus.cc:2229] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:31.715458 9710 raft_consensus.cc:2258] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:31.716070 9709 raft_consensus.cc:2258] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:31.718129 9708 tablet_replica.cc:330] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: stopping tablet replica
I20260809 22:48:31.718560 9708 raft_consensus.cc:2229] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:31.719058 9708 raft_consensus.cc:2258] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:31.720468 9709 ts_tablet_manager.cc:1905] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:31.720836 9710 ts_tablet_manager.cc:1905] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:31.722515 9708 ts_tablet_manager.cc:1905] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:31.727968 9710 ts_tablet_manager.cc:1918] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:31.728142 9710 log.cc:1196] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/f02b3c8e12804a65b5fcd4078d4cb679
I20260809 22:48:31.728710 9710 ts_tablet_manager.cc:1939] T f02b3c8e12804a65b5fcd4078d4cb679 P 8e1cb301734445c593b1d192bee7712e: Deleting consensus metadata
I20260809 22:48:31.730671 9129 catalog_manager.cc:4841] TS 8e1cb301734445c593b1d192bee7712e (127.21.27.131:33765): tablet f02b3c8e12804a65b5fcd4078d4cb679 (table test-table [id=9a1026ad349d4a97986daba62814c881]) successfully deleted
I20260809 22:48:31.730778 9708 ts_tablet_manager.cc:1918] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:31.731024 9708 log.cc:1196] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/f02b3c8e12804a65b5fcd4078d4cb679
I20260809 22:48:31.731940 9708 ts_tablet_manager.cc:1939] T f02b3c8e12804a65b5fcd4078d4cb679 P dce588b146ee45639d72c1abf654cbf4: Deleting consensus metadata
I20260809 22:48:31.734647 9129 catalog_manager.cc:4841] TS dce588b146ee45639d72c1abf654cbf4 (127.21.27.129:33641): tablet f02b3c8e12804a65b5fcd4078d4cb679 (table test-table [id=9a1026ad349d4a97986daba62814c881]) successfully deleted
I20260809 22:48:31.736181 9709 ts_tablet_manager.cc:1918] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:31.736389 9709 log.cc:1196] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/f02b3c8e12804a65b5fcd4078d4cb679
I20260809 22:48:31.737015 9709 ts_tablet_manager.cc:1939] T f02b3c8e12804a65b5fcd4078d4cb679 P 84e18855ce2e4159aeebee0cc04f228b: Deleting consensus metadata
Aug 09 22:48:31 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:48:31.739007 9129 catalog_manager.cc:4841] TS 84e18855ce2e4159aeebee0cc04f228b (127.21.27.130:46141): tablet f02b3c8e12804a65b5fcd4078d4cb679 (table test-table [id=9a1026ad349d4a97986daba62814c881]) successfully deleted
I20260809 22:48:31.745656 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.708347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44908 (local address 127.21.27.129:33641)
0809 22:48:31.708526 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.708534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.708555 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:48:31.708624 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.708632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.708732 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.708836 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:31.716243 (+ 7407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.717807 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.721313 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.721966 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.722122 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.739640 (+ 17518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:31.739655 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:31.739669 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:31.739709 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:31.742628 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.743480 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.743488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.743490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.743634 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.744067 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.744071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.744079 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.744430 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:31.744753 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.745067 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.745305 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:48:31.746351 9354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:31.840395 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9213
I20260809 22:48:31.861511 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9366
Aug 09 22:48:31 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:48:31.878895 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:31.862388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.21.27.130:46141)
0809 22:48:31.862564 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:31.862572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:31.862660 (+ 88us) server_negotiation.cc:407] Connection header received
0809 22:48:31.862884 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:31.862888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:31.862958 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:31.863089 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:31.864410 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.865241 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.866679 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:31.867128 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:31.867394 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:31.872220 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:31.872234 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:31.872250 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:31.872284 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:31.875863 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.876581 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.876601 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.876603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.876774 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:31.877203 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:31.877210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:31.877218 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:31.877603 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:31.877965 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:31.878265 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:48:31.878505 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:48:31.879674 9502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:31.993495 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9366
I20260809 22:48:32.014384 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9515
Aug 09 22:48:32 dist-test-slave-chsk krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:48:32.030118 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:32.015184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48484 (local address 127.21.27.131:33765)
0809 22:48:32.015389 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:32.015396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:32.015493 (+ 97us) server_negotiation.cc:407] Connection header received
0809 22:48:32.015718 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:32.015723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:32.015808 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:32.015922 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:32.017338 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:32.018099 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:32.019595 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:32.020003 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:32.020138 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:32.024632 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:32.024646 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:32.024662 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:32.024697 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:32.027717 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:32.028386 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:32.028393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:32.028395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:32.028543 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:32.028875 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:32.028878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:32.028880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:32.029221 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:32.029430 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:32.029691 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 22:48:32.029871 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:48:32.030807 9654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:32.120342 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9515
I20260809 22:48:32.139637 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9110
I20260809 22:48:32.140765 9178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:32.249776 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9110
2026-08-09T22:48:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4468 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T22:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:32Z Disabled control of system clock
I20260809 22:48:32.342947 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40373
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:32.673630 9721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:32.673918 9721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:32.673997 9721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:32.681133 9721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:32.681243 9721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:32.681281 9721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:32.681360 9721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:32.681404 9721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:32.690647 9721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:32.692044 9721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:32.693758 9721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:32.705547 9728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:32.705619 9727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:32.705741 9730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:32.706274 9721 server_base.cc:1034] running on GCE node
I20260809 22:48:32.707130 9721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:32.708542 9721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:32.709772 9721 hybrid_clock.cc:648] HybridClock initialized: now 1786315712709737 us; error 50 us; skew 500 ppm
I20260809 22:48:32.710127 9721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:32.712436 9721 webserver.cc:460] Webserver started at http://127.21.27.190:37765/ using document root <none> and password file <none>
I20260809 22:48:32.713133 9721 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:32.713290 9721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:32.713610 9721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:32.716496 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "60121ad3e87747cea8f4bbb55876ed13"
format_stamp: "Formatted at 2026-08-09 22:48:32 on dist-test-slave-chsk"
server_key: "557353fc91f9e6f40cae57ecb17fea67"
server_key_iv: "50ea376924002131aca1d58c641592ee"
server_key_version: "encryptionkey@0"
I20260809 22:48:32.717311 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "60121ad3e87747cea8f4bbb55876ed13"
format_stamp: "Formatted at 2026-08-09 22:48:32 on dist-test-slave-chsk"
server_key: "557353fc91f9e6f40cae57ecb17fea67"
server_key_iv: "50ea376924002131aca1d58c641592ee"
server_key_version: "encryptionkey@0"
I20260809 22:48:32.722294 9721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:48:32.725548 9738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:32.726747 9721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:48:32.726933 9721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "60121ad3e87747cea8f4bbb55876ed13"
format_stamp: "Formatted at 2026-08-09 22:48:32 on dist-test-slave-chsk"
server_key: "557353fc91f9e6f40cae57ecb17fea67"
server_key_iv: "50ea376924002131aca1d58c641592ee"
server_key_version: "encryptionkey@0"
I20260809 22:48:32.727232 9721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:32.746850 9721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:32.747700 9721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:32.747987 9721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:32.766669 9721 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40373
I20260809 22:48:32.766695 9797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40373 every 8 connection(s)
I20260809 22:48:32.767972 9721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:32.771441 9798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:32.772554 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9721
I20260809 22:48:32.772748 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:32.773180 21614 external_mini_cluster.cc:1428] Setting key 7f5979d6bbd3ccde26847dc69b55c04d
I20260809 22:48:32.785754 9798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: Bootstrap starting.
I20260809 22:48:32.788151 9801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:32.775187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36816 (local address 127.21.27.190:40373)
0809 22:48:32.776313 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:32.776332 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:32.776389 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:48:32.778103 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:32.778128 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:32.778682 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:32.779127 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:32.779680 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:32.781504 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:32.782893 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:32.784012 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:32.784172 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:32.785365 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:32.785377 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:32.785393 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:32.786099 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:32.786197 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:32.786222 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:32.786343 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:32.786555 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:48:32.786865 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":364,"threads_started":1}
I20260809 22:48:32.789444 9798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:32.790457 9798 log.cc:824] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:32.793035 9798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: No bootstrap required, opened a new log
I20260809 22:48:32.800034 9798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } }
I20260809 22:48:32.800416 9798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:32.800534 9798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60121ad3e87747cea8f4bbb55876ed13, State: Initialized, Role: FOLLOWER
I20260809 22:48:32.801060 9798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } }
I20260809 22:48:32.801312 9798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:32.801457 9798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:32.801622 9798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:32.803903 9798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } }
I20260809 22:48:32.804378 9798 leader_election.cc:302] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60121ad3e87747cea8f4bbb55876ed13; no voters:
I20260809 22:48:32.804935 9798 leader_election.cc:288] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:32.805245 9803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:32.806401 9803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [term 1 LEADER]: Becoming Leader. State: Replica: 60121ad3e87747cea8f4bbb55876ed13, State: Running, Role: LEADER
I20260809 22:48:32.807127 9798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:32.807085 9803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } }
I20260809 22:48:32.810096 9804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60121ad3e87747cea8f4bbb55876ed13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } } }
I20260809 22:48:32.810055 9805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60121ad3e87747cea8f4bbb55876ed13. Latest consensus state: current_term: 1 leader_uuid: "60121ad3e87747cea8f4bbb55876ed13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60121ad3e87747cea8f4bbb55876ed13" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40373 } } }
I20260809 22:48:32.810535 9805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:32.810537 9804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:32.817909 9811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:32.822654 9811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:32.830747 9811 catalog_manager.cc:1347] Generated new cluster ID: cea3ad0038204471b9d305b8cae35aeb
I20260809 22:48:32.830876 9811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:32.850869 9811 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:32.852382 9811 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:32.860972 9811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: Generated new TSK 0
I20260809 22:48:32.861747 9811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:32.871752 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--builtin_ntp_servers=127.21.27.148:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:33.205890 9825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:33.206147 9825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:33.206295 9825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:33.213649 9825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:33.213752 9825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:33.213933 9825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:33.223277 9825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:33.224542 9825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:33.226341 9825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:33.239365 9830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:33.239411 9831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:33.239459 9834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:33.240913 9825 server_base.cc:1034] running on GCE node
I20260809 22:48:33.241621 9825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:33.242626 9825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:33.243888 9825 hybrid_clock.cc:648] HybridClock initialized: now 1786315713243839 us; error 63 us; skew 500 ppm
I20260809 22:48:33.244246 9825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:33.246565 9825 webserver.cc:460] Webserver started at http://127.21.27.129:34121/ using document root <none> and password file <none>
I20260809 22:48:33.247318 9825 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:33.247457 9825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:33.247783 9825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:33.250586 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "cf6e6843c8134e7eb12140d417f14ae0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "e6e75904535b55aa9f7599f2c71cb614"
server_key_iv: "1d0c81748405c76681f3126d9c3eff62"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.251459 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "cf6e6843c8134e7eb12140d417f14ae0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "e6e75904535b55aa9f7599f2c71cb614"
server_key_iv: "1d0c81748405c76681f3126d9c3eff62"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.256354 9825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:48:33.259744 9841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.260948 9825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:33.261143 9825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "cf6e6843c8134e7eb12140d417f14ae0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "e6e75904535b55aa9f7599f2c71cb614"
server_key_iv: "1d0c81748405c76681f3126d9c3eff62"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.261358 9825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:33.277408 9825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:33.278190 9825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:33.278494 9825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:33.279523 9825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:33.280926 9825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:33.281049 9825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.281184 9825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:33.281292 9825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.316816 9825 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:40425
I20260809 22:48:33.316869 9969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:40425 every 8 connection(s)
I20260809 22:48:33.318329 9825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:33.322921 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9825
I20260809 22:48:33.323110 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:33.323629 21614 external_mini_cluster.cc:1428] Setting key cccd732e79717f80b55fb3d8ed369c3e
I20260809 22:48:33.327643 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--builtin_ntp_servers=127.21.27.148:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:33.331923 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:33.320913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:47803 (local address 127.21.27.190:40373)
0809 22:48:33.321396 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:33.321401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:33.322406 (+ 1005us) server_negotiation.cc:407] Connection header received
0809 22:48:33.324446 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:33.324450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:33.324516 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:33.324621 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:33.326013 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.326914 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.328627 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.329410 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.329557 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:33.330902 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:33.330911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:33.330918 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:33.330943 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:33.330997 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:33.331015 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:33.331114 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:33.331383 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 22:48:33.331497 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":276,"threads_started":1}
I20260809 22:48:33.332847 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:33.321376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40373 (local address 127.21.27.129:47803)
0809 22:48:33.322249 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:33.322295 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:33.324094 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:33.324859 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:33.324867 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:33.324949 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:33.325790 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:33.325821 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.327069 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.327073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:33.328478 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:33.328490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.329629 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.329634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:33.329830 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:33.330576 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:33.330578 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:33.330623 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:33.330702 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:33.331129 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:33.331190 (+ 61us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:33.331197 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:48:33.331343 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:48:33.331753 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":269,"threads_started":1}
I20260809 22:48:33.334416 9970 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40373
I20260809 22:48:33.334743 9970 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:33.335721 9970 heartbeater.cc:511] Master 127.21.27.190:40373 requested a full tablet report, sending...
I20260809 22:48:33.337944 9756 ts_manager.cc:194] Registered new tserver with Master: cf6e6843c8134e7eb12140d417f14ae0 (127.21.27.129:40425)
I20260809 22:48:33.339553 9756 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.129:47803
W20260809 22:48:33.666067 9974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:33.666376 9974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:33.666465 9974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:33.674019 9974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:33.674136 9974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:33.674306 9974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:33.683594 9974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:33.684847 9974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:33.686568 9974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:33.700116 9980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:33.700167 9979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:33.700191 9982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:33.701006 9974 server_base.cc:1034] running on GCE node
I20260809 22:48:33.701846 9974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:33.702965 9974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:33.704213 9974 hybrid_clock.cc:648] HybridClock initialized: now 1786315713704155 us; error 59 us; skew 500 ppm
I20260809 22:48:33.704576 9974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:33.707203 9974 webserver.cc:460] Webserver started at http://127.21.27.130:34651/ using document root <none> and password file <none>
I20260809 22:48:33.707950 9974 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:33.708097 9974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:33.708480 9974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:33.711545 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "cd1e046e6ccb4fcf862524e91692a5b0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "9ef96b558562cf73b142f2ea64ab1b4d"
server_key_iv: "72b281d096822d8148b1c051ea0555fa"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.712339 9974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "cd1e046e6ccb4fcf862524e91692a5b0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "9ef96b558562cf73b142f2ea64ab1b4d"
server_key_iv: "72b281d096822d8148b1c051ea0555fa"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.717657 9974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:48:33.721518 9988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.722896 9974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:48:33.723034 9974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "cd1e046e6ccb4fcf862524e91692a5b0"
format_stamp: "Formatted at 2026-08-09 22:48:33 on dist-test-slave-chsk"
server_key: "9ef96b558562cf73b142f2ea64ab1b4d"
server_key_iv: "72b281d096822d8148b1c051ea0555fa"
server_key_version: "encryptionkey@0"
I20260809 22:48:33.723300 9974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:33.741370 9974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:33.742532 9974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:33.742895 9974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:33.744319 9974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:33.746239 9974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:33.746332 9974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.746423 9974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:33.746480 9974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:33.783751 9974 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:40583
I20260809 22:48:33.783766 10120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:40583 every 8 connection(s)
I20260809 22:48:33.785408 9974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:33.790555 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 9974
I20260809 22:48:33.790751 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:33.791384 21614 external_mini_cluster.cc:1428] Setting key b4d3417faf48e5599b68d8c04e813167
I20260809 22:48:33.794469 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--builtin_ntp_servers=127.21.27.148:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:33.799819 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:33.788316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:57361 (local address 127.21.27.190:40373)
0809 22:48:33.788496 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:33.788500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:33.790146 (+ 1646us) server_negotiation.cc:407] Connection header received
0809 22:48:33.793138 (+ 2992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:33.793143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:33.793238 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:33.793377 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:33.794484 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.795369 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.796956 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.797373 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.797512 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:33.798943 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:33.798950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:33.798955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:33.798983 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:33.799021 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:33.799033 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:33.799132 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:33.799388 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 22:48:33.799490 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:48:33.800858 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:33.788929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40373 (local address 127.21.27.130:57361)
0809 22:48:33.789941 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:33.789984 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:33.792853 (+ 2869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:33.793622 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:33.793632 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:33.793727 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:33.794310 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:33.794328 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.795542 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.795546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:33.796792 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:33.796805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:33.797616 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:33.797623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:33.797852 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:33.798605 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:33.798607 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:33.798648 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:33.798730 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:33.799133 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:33.799164 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:33.799175 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:48:33.799378 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 22:48:33.799802 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":373,"threads_started":1}
I20260809 22:48:33.802281 10121 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40373
I20260809 22:48:33.802618 10121 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:33.803576 10121 heartbeater.cc:511] Master 127.21.27.190:40373 requested a full tablet report, sending...
I20260809 22:48:33.805174 9756 ts_manager.cc:194] Registered new tserver with Master: cd1e046e6ccb4fcf862524e91692a5b0 (127.21.27.130:40583)
I20260809 22:48:33.806008 9756 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.130:57361
W20260809 22:48:34.140062 10124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:34.140358 10124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:34.140515 10124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:34.148001 10124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:34.148109 10124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:34.148303 10124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:34.157439 10124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:34.158732 10124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:34.160534 10124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:34.173462 10130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:34.173612 10132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:34.173611 10129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:34.174543 10124 server_base.cc:1034] running on GCE node
I20260809 22:48:34.175221 10124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:34.176162 10124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:34.177363 10124 hybrid_clock.cc:648] HybridClock initialized: now 1786315714177325 us; error 29 us; skew 500 ppm
I20260809 22:48:34.177688 10124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:34.179984 10124 webserver.cc:460] Webserver started at http://127.21.27.131:42547/ using document root <none> and password file <none>
I20260809 22:48:34.180673 10124 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:34.180792 10124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:34.181128 10124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:34.184064 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "391e8edaca584726bf7c81c8a9c664ee"
format_stamp: "Formatted at 2026-08-09 22:48:34 on dist-test-slave-chsk"
server_key: "cf9d0b3d109ebc29b33c40693bb783ee"
server_key_iv: "1f0f44f4e93a8d54fbe97b5eba3ee22e"
server_key_version: "encryptionkey@0"
I20260809 22:48:34.184865 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "391e8edaca584726bf7c81c8a9c664ee"
format_stamp: "Formatted at 2026-08-09 22:48:34 on dist-test-slave-chsk"
server_key: "cf9d0b3d109ebc29b33c40693bb783ee"
server_key_iv: "1f0f44f4e93a8d54fbe97b5eba3ee22e"
server_key_version: "encryptionkey@0"
I20260809 22:48:34.189893 10124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:48:34.193295 10139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:34.194571 10124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:48:34.194749 10124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "391e8edaca584726bf7c81c8a9c664ee"
format_stamp: "Formatted at 2026-08-09 22:48:34 on dist-test-slave-chsk"
server_key: "cf9d0b3d109ebc29b33c40693bb783ee"
server_key_iv: "1f0f44f4e93a8d54fbe97b5eba3ee22e"
server_key_version: "encryptionkey@0"
I20260809 22:48:34.194965 10124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:34.237433 10124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:34.238319 10124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:34.238648 10124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:34.239835 10124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:34.241330 10124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:34.241459 10124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:34.241597 10124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:34.241693 10124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:34.278117 10124 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:33823
I20260809 22:48:34.278173 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:33823 every 8 connection(s)
I20260809 22:48:34.279497 10124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:34.288280 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10124
I20260809 22:48:34.288478 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:34.288924 21614 external_mini_cluster.cc:1428] Setting key e5b721173ab4960399166a43119da9c4
I20260809 22:48:34.291997 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.282125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:49337 (local address 127.21.27.190:40373)
0809 22:48:34.282310 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.282315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.283680 (+ 1365us) server_negotiation.cc:407] Connection header received
0809 22:48:34.285382 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.285386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.285455 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.285546 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:34.286520 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.287386 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.289068 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.289514 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.289666 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.291101 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:34.291107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:34.291112 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:34.291134 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:34.291201 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:34.291218 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:34.291314 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.291515 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.291631 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:48:34.292905 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.282625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40373 (local address 127.21.27.131:49337)
0809 22:48:34.283508 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.283554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.285134 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.285740 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.285748 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.285838 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:34.286365 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.286380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.287528 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.287532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.288916 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.288929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.289768 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.289774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.289973 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.290816 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:34.290819 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:34.290858 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:34.290942 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:34.291313 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:34.291321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:34.291327 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.291472 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.291803 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":257,"threads_started":1}
I20260809 22:48:34.294225 10273 heartbeater.cc:347] Connected to a master server at 127.21.27.190:40373
I20260809 22:48:34.294541 10273 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:34.295367 10273 heartbeater.cc:511] Master 127.21.27.190:40373 requested a full tablet report, sending...
I20260809 22:48:34.296845 9756 ts_manager.cc:194] Registered new tserver with Master: 391e8edaca584726bf7c81c8a9c664ee (127.21.27.131:33823)
I20260809 22:48:34.297681 9756 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.131:49337
I20260809 22:48:34.304617 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:34.316597 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.309112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.21.27.190:40373)
0809 22:48:34.309339 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.309348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.309823 (+ 475us) server_negotiation.cc:407] Connection header received
0809 22:48:34.310040 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.310045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.310120 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.310229 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:34.310766 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.311536 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.313540 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.314248 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.314409 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.315702 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:34.315707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:34.315710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:34.315729 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:34.315763 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:34.315775 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:34.315896 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.316076 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.316126 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:48:34.321300 9756 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:34.329005 9756 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:34.343307 9970 heartbeater.cc:503] Master 127.21.27.190:40373 was elected leader, sending a full tablet report...
I20260809 22:48:34.374310 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.341362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:40425 (local address 127.0.0.1:46660)
0809 22:48:34.365237 (+ 23875us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.365259 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.365396 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.366533 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.366541 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.366581 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:34.367043 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.367060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.368938 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.368946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.371368 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.371382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.372844 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.372850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.373531 (+ 681us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.373594 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.373723 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.373933 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23736,"mutex_wait_us":414,"thread_start_us":23492,"threads_started":1}
I20260809 22:48:34.374679 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.341189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:40583 (local address 127.0.0.1:55770)
0809 22:48:34.363219 (+ 22030us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.363287 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.363694 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.367226 (+ 3532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.367231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.367264 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:34.367712 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.367724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.369673 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.369678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.371663 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.371679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.373761 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.373769 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.373957 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.373995 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.374082 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.374214 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21545,"mutex_wait_us":5,"thread_start_us":23803,"threads_started":1}
I20260809 22:48:34.375208 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.341881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.21.27.129:40425)
0809 22:48:34.364643 (+ 22762us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.364656 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.365529 (+ 873us) server_negotiation.cc:407] Connection header received
0809 22:48:34.365648 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.365657 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.365925 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.366237 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:34.367376 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.368821 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.371582 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.372639 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.372815 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.373210 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.374606 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.374823 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22491,"thread_start_us":22316,"threads_started":1}
I20260809 22:48:34.375294 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.341381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.21.27.130:40583)
0809 22:48:34.366393 (+ 25012us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.366401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.366430 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:48:34.366524 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.366531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.366751 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.366951 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:34.368082 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.369496 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.371871 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.373243 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.373384 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.373626 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.374422 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.374896 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24830,"thread_start_us":25498,"threads_started":1}
I20260809 22:48:34.376592 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.341820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:33823 (local address 127.0.0.1:36258)
0809 22:48:34.367407 (+ 25587us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.367430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.367558 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.368275 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.368280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.368305 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:34.368629 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.368638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.370821 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.370831 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.373448 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.373462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.375798 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.375805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.375967 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.376009 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.376123 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.376255 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25446,"mutex_wait_us":28,"thread_start_us":23735,"threads_started":1}
I20260809 22:48:34.376727 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.342616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.21.27.131:33823)
0809 22:48:34.362035 (+ 19419us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.362050 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.367728 (+ 5678us) server_negotiation.cc:407] Connection header received
0809 22:48:34.367836 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.367845 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.368062 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.368260 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:34.368817 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.370652 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.373677 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.375605 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.375743 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.375979 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.376175 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.376341 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19093,"thread_start_us":19453,"threads_started":1}
I20260809 22:48:34.379065 9898 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (DEFAULT_TABLE table=test-table [id=5f1876a055f9484fabecb511c04006d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:34.379133 10047 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (DEFAULT_TABLE table=test-table [id=5f1876a055f9484fabecb511c04006d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:34.380466 9898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1d008131854a3bb59d9a8bc8a1acf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:34.380511 10047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1d008131854a3bb59d9a8bc8a1acf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:34.380208 10197 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (DEFAULT_TABLE table=test-table [id=5f1876a055f9484fabecb511c04006d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:34.381462 10197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1d008131854a3bb59d9a8bc8a1acf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:34.392274 10291 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Bootstrap starting.
I20260809 22:48:34.394905 10292 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Bootstrap starting.
I20260809 22:48:34.396571 10293 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Bootstrap starting.
I20260809 22:48:34.398016 10291 tablet_bootstrap.cc:654] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:34.398523 10292 tablet_bootstrap.cc:654] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:34.399416 10291 log.cc:824] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:34.399631 10293 tablet_bootstrap.cc:654] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:34.399856 10292 log.cc:824] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:34.400781 10293 log.cc:824] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:34.402617 10291 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: No bootstrap required, opened a new log
I20260809 22:48:34.402730 10292 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: No bootstrap required, opened a new log
I20260809 22:48:34.402897 10291 ts_tablet_manager.cc:1397] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260809 22:48:34.402998 10292 ts_tablet_manager.cc:1397] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:48:34.403728 10293 tablet_bootstrap.cc:492] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: No bootstrap required, opened a new log
I20260809 22:48:34.403965 10293 ts_tablet_manager.cc:1397] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:48:34.411273 10292 raft_consensus.cc:348] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.411695 10292 raft_consensus.cc:374] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:34.411806 10292 raft_consensus.cc:729] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1e046e6ccb4fcf862524e91692a5b0, State: Initialized, Role: FOLLOWER
I20260809 22:48:34.412475 10292 consensus_queue.cc:260] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.413676 10121 heartbeater.cc:503] Master 127.21.27.190:40373 was elected leader, sending a full tablet report...
I20260809 22:48:34.413641 10291 raft_consensus.cc:348] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.414060 10291 raft_consensus.cc:374] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:34.414151 10291 raft_consensus.cc:729] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf6e6843c8134e7eb12140d417f14ae0, State: Initialized, Role: FOLLOWER
I20260809 22:48:34.414358 10292 ts_tablet_manager.cc:1428] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 22:48:34.414614 10293 raft_consensus.cc:348] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.414752 10291 consensus_queue.cc:260] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.415005 10293 raft_consensus.cc:374] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:34.415110 10293 raft_consensus.cc:729] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 391e8edaca584726bf7c81c8a9c664ee, State: Initialized, Role: FOLLOWER
I20260809 22:48:34.415799 10293 consensus_queue.cc:260] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.421025 10297 raft_consensus.cc:482] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:34.421633 10297 raft_consensus.cc:504] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.423825 10291 ts_tablet_manager.cc:1428] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260809 22:48:34.424911 10297 leader_election.cc:288] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf6e6843c8134e7eb12140d417f14ae0 (127.21.27.129:40425), 391e8edaca584726bf7c81c8a9c664ee (127.21.27.131:33823)
I20260809 22:48:34.431325 10273 heartbeater.cc:503] Master 127.21.27.190:40373 was elected leader, sending a full tablet report...
I20260809 22:48:34.432075 10293 ts_tablet_manager.cc:1428] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Time spent starting tablet: real 0.028s user 0.023s sys 0.004s
I20260809 22:48:34.439774 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.426112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:52377 (local address 127.21.27.131:33823)
0809 22:48:34.426622 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.426628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.433440 (+ 6812us) server_negotiation.cc:407] Connection header received
0809 22:48:34.433642 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.433650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.433715 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.433773 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:34.434666 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.435693 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.437891 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.438770 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.438891 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.438938 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.439374 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.439513 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260809 22:48:34.439801 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.425400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:33823 (local address 127.21.27.130:52377)
0809 22:48:34.433314 (+ 7914us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.433333 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.433460 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.434028 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.434032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.434062 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:34.434499 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.434513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.435852 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.435857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.437694 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.437706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.438967 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.438977 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.439198 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.439235 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.439310 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.439427 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7788,"thread_start_us":7149,"threads_started":1}
I20260809 22:48:34.439877 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.425540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:53831 (local address 127.21.27.129:40425)
0809 22:48:34.425797 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.425806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.432977 (+ 7171us) server_negotiation.cc:407] Connection header received
0809 22:48:34.433333 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.433338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.433439 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.433528 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:34.434180 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.435240 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.438094 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.438890 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.439046 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.439115 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.439498 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.439644 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:48:34.439884 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.424962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:40425 (local address 127.21.27.130:53831)
0809 22:48:34.432868 (+ 7906us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:34.432889 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:34.433126 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:34.433581 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:34.433588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:34.433624 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:34.433992 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.434003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.435355 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.435361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.437901 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:34.437916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.439083 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.439087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:34.439308 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.439341 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:48:34.439438 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:48:34.439554 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7728,"thread_start_us":8009,"threads_started":1}
I20260809 22:48:34.440958 9921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1d008131854a3bb59d9a8bc8a1acf9" candidate_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6e6843c8134e7eb12140d417f14ae0" is_pre_election: true
I20260809 22:48:34.441111 10222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1d008131854a3bb59d9a8bc8a1acf9" candidate_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "391e8edaca584726bf7c81c8a9c664ee" is_pre_election: true
I20260809 22:48:34.441546 9921 raft_consensus.cc:2454] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1e046e6ccb4fcf862524e91692a5b0 in term 0.
I20260809 22:48:34.441612 10222 raft_consensus.cc:2454] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd1e046e6ccb4fcf862524e91692a5b0 in term 0.
I20260809 22:48:34.442297 9994 leader_election.cc:302] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 391e8edaca584726bf7c81c8a9c664ee, cd1e046e6ccb4fcf862524e91692a5b0; no voters:
I20260809 22:48:34.442728 10297 raft_consensus.cc:2781] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:34.442869 10297 raft_consensus.cc:482] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:34.442972 10297 raft_consensus.cc:3037] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:34.445284 10297 raft_consensus.cc:504] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.445993 10297 leader_election.cc:288] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [CANDIDATE]: Term 1 election: Requested vote from peers cf6e6843c8134e7eb12140d417f14ae0 (127.21.27.129:40425), 391e8edaca584726bf7c81c8a9c664ee (127.21.27.131:33823)
I20260809 22:48:34.446394 9921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1d008131854a3bb59d9a8bc8a1acf9" candidate_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6e6843c8134e7eb12140d417f14ae0"
I20260809 22:48:34.446532 10222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1d008131854a3bb59d9a8bc8a1acf9" candidate_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "391e8edaca584726bf7c81c8a9c664ee"
I20260809 22:48:34.446614 9921 raft_consensus.cc:3037] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:34.446765 10222 raft_consensus.cc:3037] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:34.448999 9921 raft_consensus.cc:2454] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1e046e6ccb4fcf862524e91692a5b0 in term 1.
I20260809 22:48:34.449002 10222 raft_consensus.cc:2454] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd1e046e6ccb4fcf862524e91692a5b0 in term 1.
I20260809 22:48:34.449426 9994 leader_election.cc:302] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 391e8edaca584726bf7c81c8a9c664ee, cd1e046e6ccb4fcf862524e91692a5b0, cf6e6843c8134e7eb12140d417f14ae0; no voters:
I20260809 22:48:34.449757 10297 raft_consensus.cc:2781] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:34.449928 10297 raft_consensus.cc:686] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 1 LEADER]: Becoming Leader. State: Replica: cd1e046e6ccb4fcf862524e91692a5b0, State: Running, Role: LEADER
I20260809 22:48:34.450424 10297 consensus_queue.cc:237] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } }
I20260809 22:48:34.455044 9756 catalog_manager.cc:5495] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd1e046e6ccb4fcf862524e91692a5b0 (127.21.27.130). New cstate: current_term: 1 leader_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1e046e6ccb4fcf862524e91692a5b0" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 40583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 } health_report { overall_health: UNKNOWN } } }
I20260809 22:48:34.484753 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.479189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.21.27.130:40583)
0809 22:48:34.479395 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.479404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.479430 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:48:34.479689 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.479693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.479789 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.479945 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:34.480334 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.481030 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.482440 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.482860 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.483002 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.483646 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:34.484112 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:34.484300 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.484471 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.484514 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:48:34.492496 10222 raft_consensus.cc:1261] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 1 FOLLOWER]: Refusing update from remote peer cd1e046e6ccb4fcf862524e91692a5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:34.492497 9921 raft_consensus.cc:1261] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 1 FOLLOWER]: Refusing update from remote peer cd1e046e6ccb4fcf862524e91692a5b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:34.493481 10300 consensus_queue.cc:1035] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf6e6843c8134e7eb12140d417f14ae0" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 40425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:34.493819 10297 consensus_queue.cc:1035] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "391e8edaca584726bf7c81c8a9c664ee" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 33823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:34.518743 9755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:34.519172 9755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:34.522985 9755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60121ad3e87747cea8f4bbb55876ed13: Sending DeleteTablet for 3 replicas of tablet 3e1d008131854a3bb59d9a8bc8a1acf9
I20260809 22:48:34.524214 9898 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:34 UTC) from {username='slave'} at 127.0.0.1:46660
I20260809 22:48:34.524230 10047 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:34 UTC) from {username='slave'} at 127.0.0.1:55770
I20260809 22:48:34.525122 10318 tablet_replica.cc:330] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: stopping tablet replica
I20260809 22:48:34.525626 10318 raft_consensus.cc:2229] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:34.525738 10197 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1d008131854a3bb59d9a8bc8a1acf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:34 UTC) from {username='slave'} at 127.0.0.1:36258
I20260809 22:48:34.526296 10318 raft_consensus.cc:2258] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:34.527508 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9825
I20260809 22:48:34.528527 10318 ts_tablet_manager.cc:1905] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:34.533937 10318 ts_tablet_manager.cc:1918] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:34.534111 10318 log.cc:1196] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/3e1d008131854a3bb59d9a8bc8a1acf9
I20260809 22:48:34.534611 10318 ts_tablet_manager.cc:1939] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cf6e6843c8134e7eb12140d417f14ae0: Deleting consensus metadata
I20260809 22:48:34.536203 9739 catalog_manager.cc:4841] TS cf6e6843c8134e7eb12140d417f14ae0 (127.21.27.129:40425): tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (table test-table [id=5f1876a055f9484fabecb511c04006d6]) successfully deleted
I20260809 22:48:34.536541 10319 tablet_replica.cc:330] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: stopping tablet replica
I20260809 22:48:34.537097 10319 raft_consensus.cc:2229] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:34.537915 10319 raft_consensus.cc:2258] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:34.540606 10319 ts_tablet_manager.cc:1905] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:34.541133 10320 tablet_replica.cc:330] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: stopping tablet replica
I20260809 22:48:34.541548 10320 raft_consensus.cc:2229] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:34.542073 10320 raft_consensus.cc:2258] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:34.544124 10320 ts_tablet_manager.cc:1905] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:34.545354 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.528506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.21.27.129:40425)
0809 22:48:34.528702 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.528709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.535994 (+ 7285us) server_negotiation.cc:407] Connection header received
0809 22:48:34.536579 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.536584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.536676 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.536782 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:34.537570 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.538712 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.540728 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.541359 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.541462 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.543565 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:34.543577 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:34.543594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:34.544343 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:34.544402 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:34.544426 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:34.544516 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.544830 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.544950 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:48:34.546742 9965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:34.548154 10319 ts_tablet_manager.cc:1918] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:34.548390 10319 log.cc:1196] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/3e1d008131854a3bb59d9a8bc8a1acf9
I20260809 22:48:34.549026 10319 ts_tablet_manager.cc:1939] T 3e1d008131854a3bb59d9a8bc8a1acf9 P cd1e046e6ccb4fcf862524e91692a5b0: Deleting consensus metadata
I20260809 22:48:34.551316 9740 catalog_manager.cc:4841] TS cd1e046e6ccb4fcf862524e91692a5b0 (127.21.27.130:40583): tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (table test-table [id=5f1876a055f9484fabecb511c04006d6]) successfully deleted
I20260809 22:48:34.552413 10320 ts_tablet_manager.cc:1918] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:34.552577 10320 log.cc:1196] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/3e1d008131854a3bb59d9a8bc8a1acf9
I20260809 22:48:34.553059 10320 ts_tablet_manager.cc:1939] T 3e1d008131854a3bb59d9a8bc8a1acf9 P 391e8edaca584726bf7c81c8a9c664ee: Deleting consensus metadata
I20260809 22:48:34.554875 9742 catalog_manager.cc:4841] TS 391e8edaca584726bf7c81c8a9c664ee (127.21.27.131:33823): tablet 3e1d008131854a3bb59d9a8bc8a1acf9 (table test-table [id=5f1876a055f9484fabecb511c04006d6]) successfully deleted
I20260809 22:48:34.640548 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9825
I20260809 22:48:34.660908 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9974
I20260809 22:48:34.669564 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.661767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.21.27.130:40583)
0809 22:48:34.661973 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.661978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.662195 (+ 217us) server_negotiation.cc:407] Connection header received
0809 22:48:34.662415 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.662420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.662544 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.662631 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:34.663216 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.664081 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.666011 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.666481 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.666585 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.667973 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:34.667982 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:34.667997 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:34.668707 (+ 710us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:34.668756 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:34.668779 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:34.668873 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.669067 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.669154 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:48:34.670392 10115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:34.783721 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9974
I20260809 22:48:34.805344 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10124
I20260809 22:48:34.812135 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:34.806119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.21.27.131:33823)
0809 22:48:34.806313 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:34.806318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:34.806338 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:48:34.806472 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:34.806476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:34.806554 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:34.806641 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:34.806991 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.807851 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.809201 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:34.809581 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:34.809677 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:34.810611 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:34.810620 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:34.810634 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:34.811363 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:34.811415 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:34.811435 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:34.811524 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:34.811696 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 22:48:34.811781 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:48:34.812845 10266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:34.901408 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10124
I20260809 22:48:34.920859 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 9721
I20260809 22:48:34.921943 9792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:35.031772 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 9721
2026-08-09T22:48:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2780 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:35 dist-test-slave-chsk krb5kdc[10329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:35 dist-test-slave-chsk krb5kdc[10329](info): set up 2 sockets
Aug 09 22:48:35 dist-test-slave-chsk krb5kdc[10329](info): commencing operation
krb5kdc: starting...
W20260809 22:48:36.216367 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:36 dist-test-slave-chsk krb5kdc[10329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:48:36.530339 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45647
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:36.871685 10342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:36.872026 10342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:36.872213 10342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:36.879647 10342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:36.879804 10342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:36.879905 10342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:36.880010 10342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:36.880092 10342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:36.889843 10342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:36.891258 10342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:36.892618 10342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:36.907676 10347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:36.907688 10351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:36.907727 10348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:36.908623 10342 server_base.cc:1034] running on GCE node
I20260809 22:48:36.909359 10342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:36.910791 10342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:36.911998 10342 hybrid_clock.cc:648] HybridClock initialized: now 1786315716911945 us; error 48 us; skew 500 ppm
I20260809 22:48:36.912367 10342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:36 dist-test-slave-chsk krb5kdc[10329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:36.919567 10342 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:48:36.921260 10342 webserver.cc:460] Webserver started at http://127.21.27.190:41481/ using document root <none> and password file <none>
I20260809 22:48:36.921984 10342 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:36.922133 10342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:36.922473 10342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:36.925619 10342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "28b67c4cc5ff4655984137213f36b08d"
format_stamp: "Formatted at 2026-08-09 22:48:36 on dist-test-slave-chsk"
server_key: "181d998a47dc25404441d13c7611b28c"
server_key_iv: "33b881f125170fbc0727e7ecf6b43302"
server_key_version: "encryptionkey@0"
I20260809 22:48:36.926416 10342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "28b67c4cc5ff4655984137213f36b08d"
format_stamp: "Formatted at 2026-08-09 22:48:36 on dist-test-slave-chsk"
server_key: "181d998a47dc25404441d13c7611b28c"
server_key_iv: "33b881f125170fbc0727e7ecf6b43302"
server_key_version: "encryptionkey@0"
I20260809 22:48:36.931768 10342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:48:36.935382 10358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:36.936676 10342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:36.936869 10342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "28b67c4cc5ff4655984137213f36b08d"
format_stamp: "Formatted at 2026-08-09 22:48:36 on dist-test-slave-chsk"
server_key: "181d998a47dc25404441d13c7611b28c"
server_key_iv: "33b881f125170fbc0727e7ecf6b43302"
server_key_version: "encryptionkey@0"
I20260809 22:48:36.937080 10342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:36.953760 10342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:36.957871 10342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:36.958177 10342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:36.977468 10342 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45647
I20260809 22:48:36.977492 10415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45647 every 8 connection(s)
I20260809 22:48:36.978889 10342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:36.981015 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10342
I20260809 22:48:36.981222 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:36.981813 21614 external_mini_cluster.cc:1428] Setting key 3237b3a06df60f6a6e6bfb165c3b98a6
I20260809 22:48:36.983541 10416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:48:36 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:36.997654 10416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: Bootstrap starting.
I20260809 22:48:37.000909 10416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:37.002332 10416 log.cc:824] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:37.005553 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:36.983819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.21.27.190:45647)
0809 22:48:36.984619 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:36.984630 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:36.984669 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:48:36.986297 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:36.986317 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:36.986684 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:36.987001 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:36.988943 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:36.990277 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:36.992105 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:36.992754 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:36.992879 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:36.997759 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:36.997768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:36.997804 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:36.997867 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:37.001045 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:37.001972 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:37.001978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:37.001984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:37.002167 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:37.002626 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:37.002630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:37.002632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:37.003454 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:37.003824 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:37.004163 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 22:48:37.004519 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":382,"threads_started":1}
I20260809 22:48:37.005668 10416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: No bootstrap required, opened a new log
I20260809 22:48:37.013161 10416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } }
I20260809 22:48:37.013556 10416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:37.013674 10416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b67c4cc5ff4655984137213f36b08d, State: Initialized, Role: FOLLOWER
I20260809 22:48:37.014218 10416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } }
I20260809 22:48:37.014468 10416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:37.014621 10416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:37.014797 10416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:37.017246 10416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } }
I20260809 22:48:37.017737 10416 leader_election.cc:302] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28b67c4cc5ff4655984137213f36b08d; no voters:
I20260809 22:48:37.018358 10416 leader_election.cc:288] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:37.018500 10421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:37.019218 10421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [term 1 LEADER]: Becoming Leader. State: Replica: 28b67c4cc5ff4655984137213f36b08d, State: Running, Role: LEADER
I20260809 22:48:37.019762 10421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } }
I20260809 22:48:37.020490 10416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:37.022528 10422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28b67c4cc5ff4655984137213f36b08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } } }
I20260809 22:48:37.023005 10422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:37.024163 10427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:37.024415 10423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28b67c4cc5ff4655984137213f36b08d. Latest consensus state: current_term: 1 leader_uuid: "28b67c4cc5ff4655984137213f36b08d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28b67c4cc5ff4655984137213f36b08d" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45647 } } }
I20260809 22:48:37.024763 10423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:37.030736 10427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:37.044521 10440 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:48:37.045845 10427 catalog_manager.cc:1347] Generated new cluster ID: d91f8946d8744831a1252315772a3f4d
I20260809 22:48:37.045972 10427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:37.071496 10427 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:37.073452 10427 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:37.084499 10427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: Generated new TSK 0
I20260809 22:48:37.085383 10427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.27.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
Entry for principal kudu/127.21.27.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab.
I20260809 22:48:37.168646 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:37.516090 10443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:37.516423 10443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:37.516603 10443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:37.524618 10443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:37.524778 10443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:37.524972 10443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:37.534894 10443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.129.keytab
--principal=kudu/127.21.27.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:37.536279 10443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:37.537586 10443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:37.551342 10453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:37.551391 10450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:37.551306 10449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:37.552523 10443 server_base.cc:1034] running on GCE node
I20260809 22:48:37.553120 10443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:37.554020 10443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:37.555270 10443 hybrid_clock.cc:648] HybridClock initialized: now 1786315717555205 us; error 65 us; skew 500 ppm
I20260809 22:48:37.555652 10443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:37 dist-test-slave-chsk krb5kdc[10329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:37.558815 10443 init.cc:377] Logged in from keytab as kudu/127.21.27.129@KRBTEST.COM (short username kudu)
I20260809 22:48:37.560431 10443 webserver.cc:460] Webserver started at http://127.21.27.129:45755/ using document root <none> and password file <none>
I20260809 22:48:37.561091 10443 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:37.561249 10443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:37.561564 10443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:37.564447 10443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "c74d849b9a34466a95c9bb6759e96070"
format_stamp: "Formatted at 2026-08-09 22:48:37 on dist-test-slave-chsk"
server_key: "1d43205e6701b508a63b1fdca024a64f"
server_key_iv: "0197abc2dc30e4bf58cb8dc271335034"
server_key_version: "encryptionkey@0"
I20260809 22:48:37.565210 10443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "c74d849b9a34466a95c9bb6759e96070"
format_stamp: "Formatted at 2026-08-09 22:48:37 on dist-test-slave-chsk"
server_key: "1d43205e6701b508a63b1fdca024a64f"
server_key_iv: "0197abc2dc30e4bf58cb8dc271335034"
server_key_version: "encryptionkey@0"
I20260809 22:48:37.570236 10443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:48:37.573443 10461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:37.574648 10443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:48:37.574836 10443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "c74d849b9a34466a95c9bb6759e96070"
format_stamp: "Formatted at 2026-08-09 22:48:37 on dist-test-slave-chsk"
server_key: "1d43205e6701b508a63b1fdca024a64f"
server_key_iv: "0197abc2dc30e4bf58cb8dc271335034"
server_key_version: "encryptionkey@0"
I20260809 22:48:37.575047 10443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:37.595185 10443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:37.599050 10443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:37.599373 10443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:37.600310 10443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:37.601595 10443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:37.601696 10443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:37.601852 10443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:37.601925 10443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:37.635999 10443 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:39221
I20260809 22:48:37.636051 10582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:39221 every 8 connection(s)
I20260809 22:48:37.637373 10443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:37.639760 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10443
I20260809 22:48:37.639968 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:37.640529 21614 external_mini_cluster.cc:1428] Setting key 37690a744d2b9f228c1135f68a0e8c65
Aug 09 22:48:37 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315717, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.129@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:37.661710 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:37.640007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:56577 (local address 127.21.27.190:45647)
0809 22:48:37.640515 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:37.640521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:37.641689 (+ 1168us) server_negotiation.cc:407] Connection header received
0809 22:48:37.644198 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:37.644202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:37.644262 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:37.644347 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:37.646477 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:37.647574 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:37.650021 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:37.650665 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:37.650791 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:37.655403 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:37.655412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:37.655420 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:37.655464 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:37.658268 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:37.659277 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:37.659281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:37.659284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:37.659407 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:37.659976 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:37.659979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:37.659981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:37.660238 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:37.660349 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:37.661280 (+ 931us) server_negotiation.cc:299] Negotiation successful
0809 22:48:37.661446 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":253,"threads_started":1}
I20260809 22:48:37.663208 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:37.640638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45647 (local address 127.21.27.129:56577)
0809 22:48:37.641486 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:37.641558 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:37.643928 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:37.644635 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:37.644650 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:37.645486 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:37.646270 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:37.646298 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:37.647739 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:37.647747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:37.649861 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:37.649879 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:37.650914 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:37.650923 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:37.651303 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:37.652091 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:37.652132 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:37.655135 (+ 3003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:37.658431 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:37.658443 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:37.658461 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:37.659117 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:37.659534 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:37.659542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:37.659549 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:37.659843 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:37.660373 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:37.660383 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:37.660857 (+ 474us) client_negotiation.cc:769] Sending connection context
0809 22:48:37.661167 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 22:48:37.661717 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":343,"threads_started":1}
I20260809 22:48:37.664903 10583 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45647
I20260809 22:48:37.665223 10583 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:37.666160 10583 heartbeater.cc:511] Master 127.21.27.190:45647 requested a full tablet report, sending...
I20260809 22:48:37.668660 10376 ts_manager.cc:194] Registered new tserver with Master: c74d849b9a34466a95c9bb6759e96070 (127.21.27.129:39221)
I20260809 22:48:37.670913 10376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.129@KRBTEST.COM'} at 127.21.27.129:56577
WARNING: no policy specified for kudu/127.21.27.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
Entry for principal kudu/127.21.27.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab.
I20260809 22:48:37.710213 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:38.037338 10590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:38.037600 10590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:38.037703 10590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:38.044945 10590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:38.045086 10590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:38.045215 10590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:38.054358 10590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.130.keytab
--principal=kudu/127.21.27.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:38.055495 10590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:38.056684 10590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:38.069614 10596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:38.069674 10595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:38.069640 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:38.070387 10590 server_base.cc:1034] running on GCE node
I20260809 22:48:38.070940 10590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:38.071835 10590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:38.073037 10590 hybrid_clock.cc:648] HybridClock initialized: now 1786315718073000 us; error 40 us; skew 500 ppm
I20260809 22:48:38.073338 10590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:38 dist-test-slave-chsk krb5kdc[10329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:38.076547 10590 init.cc:377] Logged in from keytab as kudu/127.21.27.130@KRBTEST.COM (short username kudu)
I20260809 22:48:38.078147 10590 webserver.cc:460] Webserver started at http://127.21.27.130:43395/ using document root <none> and password file <none>
I20260809 22:48:38.078886 10590 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:38.079031 10590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:38.079484 10590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:38.082417 10590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "32302d316ae7423090fc043bba70849d"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "caae599643390fe6dfd348cbf2894427"
server_key_iv: "1799eae4f1fbe1d272d7958d5d52bb1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.083348 10590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "32302d316ae7423090fc043bba70849d"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "caae599643390fe6dfd348cbf2894427"
server_key_iv: "1799eae4f1fbe1d272d7958d5d52bb1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.088537 10590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 22:48:38.091910 10606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.093164 10590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:38.093338 10590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "32302d316ae7423090fc043bba70849d"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "caae599643390fe6dfd348cbf2894427"
server_key_iv: "1799eae4f1fbe1d272d7958d5d52bb1d"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.093539 10590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:38.119073 10590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:38.123667 10590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:38.123962 10590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:38.124959 10590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:38.126291 10590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:38.126397 10590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.126551 10590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:38.126626 10590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.160687 10590 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:34061
I20260809 22:48:38.160737 10731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:34061 every 8 connection(s)
I20260809 22:48:38.162046 10590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:38.165858 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10590
I20260809 22:48:38.166043 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:38.166604 21614 external_mini_cluster.cc:1428] Setting key e08473bc691325ccf5f962e1d8a36e0d
Aug 09 22:48:38 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.130@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:38.184100 10734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.164600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:51375 (local address 127.21.27.190:45647)
0809 22:48:38.165037 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.165042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.167241 (+ 2199us) server_negotiation.cc:407] Connection header received
0809 22:48:38.168074 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.168080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.168172 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.168295 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:38.169805 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.171022 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.172605 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.173357 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.173504 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.177129 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:38.177134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:38.177137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:38.177182 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:38.180335 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.181102 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.181109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.181116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.181293 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.181868 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.181875 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.181879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.182312 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:38.182479 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.183326 (+ 847us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.183594 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":210,"threads_started":1}
I20260809 22:48:38.184731 10735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.165155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45647 (local address 127.21.27.130:51375)
0809 22:48:38.165842 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.165882 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.167822 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.168565 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.168573 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.169075 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:38.169620 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.169636 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.171238 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.171242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.172437 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.172453 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.173578 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.173583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.173772 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.174484 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:38.174514 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:38.176882 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:38.180486 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:38.180495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:38.180511 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:38.180956 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:38.181517 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:38.181521 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:38.181524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:38.181718 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:38.182440 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:38.182446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:38.182798 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.183104 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.183582 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":302,"threads_started":1}
I20260809 22:48:38.186297 10732 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45647
I20260809 22:48:38.186662 10732 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:38.187557 10732 heartbeater.cc:511] Master 127.21.27.190:45647 requested a full tablet report, sending...
I20260809 22:48:38.189182 10376 ts_manager.cc:194] Registered new tserver with Master: 32302d316ae7423090fc043bba70849d (127.21.27.130:34061)
I20260809 22:48:38.190018 10376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.130@KRBTEST.COM'} at 127.21.27.130:51375
WARNING: no policy specified for kudu/127.21.27.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
Entry for principal kudu/127.21.27.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab.
I20260809 22:48:38.235960 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:43105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:38.563710 10739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:38.563969 10739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:38.564136 10739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:38.571204 10739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:38.571310 10739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:38.571506 10739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:38.581403 10739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:43105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.131.keytab
--principal=kudu/127.21.27.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:38.582715 10739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:38.584164 10739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:38.597504 10745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:38.597435 10750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:38.597476 10747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:38.598414 10739 server_base.cc:1034] running on GCE node
I20260809 22:48:38.599056 10739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:38.600033 10739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:38.601233 10739 hybrid_clock.cc:648] HybridClock initialized: now 1786315718601172 us; error 63 us; skew 500 ppm
I20260809 22:48:38.601583 10739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:38 dist-test-slave-chsk krb5kdc[10329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:38.604823 10739 init.cc:377] Logged in from keytab as kudu/127.21.27.131@KRBTEST.COM (short username kudu)
I20260809 22:48:38.606199 10739 webserver.cc:460] Webserver started at http://127.21.27.131:32983/ using document root <none> and password file <none>
I20260809 22:48:38.606825 10739 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:38.606927 10739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:38.607290 10739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:38.610078 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "ca353b3a6dec4f66a2019a9ee5366873"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "819bf24ad706b8a9e8e32fcc725ac5e5"
server_key_iv: "b9ed43317a6a2a6f1d48945af9cdd9f3"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.610775 10739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "ca353b3a6dec4f66a2019a9ee5366873"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "819bf24ad706b8a9e8e32fcc725ac5e5"
server_key_iv: "b9ed43317a6a2a6f1d48945af9cdd9f3"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.615823 10739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 22:48:38.619040 10758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.620364 10739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:38.620532 10739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "ca353b3a6dec4f66a2019a9ee5366873"
format_stamp: "Formatted at 2026-08-09 22:48:38 on dist-test-slave-chsk"
server_key: "819bf24ad706b8a9e8e32fcc725ac5e5"
server_key_iv: "b9ed43317a6a2a6f1d48945af9cdd9f3"
server_key_version: "encryptionkey@0"
I20260809 22:48:38.620754 10739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:38.644306 10739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:38.648476 10739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:38.648736 10739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:38.649621 10739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:38.650820 10739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:38.650892 10739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.650970 10739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:38.651013 10739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:38.674731 10583 heartbeater.cc:503] Master 127.21.27.190:45647 was elected leader, sending a full tablet report...
I20260809 22:48:38.685081 10739 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:40217
I20260809 22:48:38.685168 10879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:40217 every 8 connection(s)
I20260809 22:48:38.686422 10739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:38.692901 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10739
I20260809 22:48:38.693086 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:38.693535 21614 external_mini_cluster.cc:1428] Setting key abb1d860fd2c9283c2c905e65870efcf
Aug 09 22:48:38 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315718, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.131@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:48:38.705639 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.688872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:42005 (local address 127.21.27.190:45647)
0809 22:48:38.689370 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.689377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.690294 (+ 917us) server_negotiation.cc:407] Connection header received
0809 22:48:38.692132 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.692136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.692214 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.692310 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:38.693915 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.694898 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.696649 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.697128 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.697226 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.700491 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:38.700496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:38.700499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:38.700530 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:38.702964 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.703663 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.703667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.703669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.703787 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.704154 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.704157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.704159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.704446 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:38.704567 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.705186 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.705361 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":221,"threads_started":1}
I20260809 22:48:38.706559 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.689460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45647 (local address 127.21.27.131:42005)
0809 22:48:38.690145 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.690181 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.691886 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.692529 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.692536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.693059 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:38.693738 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.693756 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.695040 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.695044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.696523 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.696534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.697309 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.697314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.697492 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.698188 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:48:38.698209 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:38.700307 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:38.703085 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:38.703091 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:38.703101 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:38.703555 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:38.703878 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:48:38.703881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:48:38.703883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:48:38.704053 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:48:38.704561 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:38.704566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:38.704854 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.705073 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.705440 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":268,"threads_started":1}
I20260809 22:48:38.708034 10880 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45647
I20260809 22:48:38.708388 10880 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:38.709177 10880 heartbeater.cc:511] Master 127.21.27.190:45647 requested a full tablet report, sending...
I20260809 22:48:38.710541 10376 ts_manager.cc:194] Registered new tserver with Master: ca353b3a6dec4f66a2019a9ee5366873 (127.21.27.131:40217)
I20260809 22:48:38.711463 10376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.27.131@KRBTEST.COM'} at 127.21.27.131:42005
I20260809 22:48:38.721100 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:38.738543 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.725441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38710 (local address 127.21.27.190:45647)
0809 22:48:38.725640 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.725648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.726030 (+ 382us) server_negotiation.cc:407] Connection header received
0809 22:48:38.726243 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.726252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.726314 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.726415 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:38.727897 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.728668 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.729973 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.730447 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.730579 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.733044 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:38.733057 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:38.733060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:38.733092 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:38.736003 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.736657 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.736661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.736663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.736791 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:38.737147 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:38.737152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:38.737154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:38.737479 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:38.737673 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.737971 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.738149 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:48:38.742520 10376 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:38.749445 10376 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:38.772065 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.762299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.21.27.130:34061)
0809 22:48:38.763554 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.763562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.763588 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:48:38.763681 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.763687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.763922 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.765114 (+ 1192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:38.765387 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.766682 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.769593 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.770909 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.771064 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.771279 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.771587 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.771763 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1078,"thread_start_us":974,"threads_started":1}
I20260809 22:48:38.772149 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.761875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:34061 (local address 127.0.0.1:40508)
0809 22:48:38.762785 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.762809 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.762957 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.764189 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.764197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.764292 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:38.764864 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.764881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.766853 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.766861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.769438 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.769450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.771102 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.771107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.771326 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.771377 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.771526 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.771708 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":28,"thread_start_us":375,"threads_started":1}
I20260809 22:48:38.772861 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.761251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.21.27.131:40217)
0809 22:48:38.762082 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.762095 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.762771 (+ 676us) server_negotiation.cc:407] Connection header received
0809 22:48:38.763193 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.763201 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.763446 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.763664 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:38.765587 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.767187 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.770070 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.771452 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.771599 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.771797 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.772397 (+ 600us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.772552 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":426,"threads_started":1}
I20260809 22:48:38.773661 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.761095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:40217 (local address 127.0.0.1:42198)
0809 22:48:38.762620 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.762676 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.762919 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.764095 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.764106 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.764143 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:38.765365 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.765380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.767318 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.767326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.769899 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.769914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.771941 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.771948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.772169 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.772215 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.773110 (+ 895us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.773261 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":22,"spinlock_wait_cycles":1699584,"thread_start_us":1723,"threads_started":1}
I20260809 22:48:38.773762 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.763130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:39221 (local address 127.0.0.1:43204)
0809 22:48:38.764706 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.764723 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.764849 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.766682 (+ 1833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.766687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.766714 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:38.767220 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.767233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.769196 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.769201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.770827 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.770839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.772951 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.772959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.773146 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.773186 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.773272 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.773379 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"mutex_wait_us":4,"thread_start_us":1256,"threads_started":1}
I20260809 22:48:38.774474 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.763663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.21.27.129:39221)
0809 22:48:38.766169 (+ 2506us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.766177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.766203 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:48:38.766290 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.766296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.766480 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.766668 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:38.767443 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.769009 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.770981 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.772458 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.772590 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.772804 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.773963 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.774154 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2322,"thread_start_us":2159,"threads_started":1}
I20260809 22:48:38.776793 10810 tablet_service.cc:1467] Processing CreateTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 (DEFAULT_TABLE table=test-table [id=39cc64e556c14fbaa0e70fe078decb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:38.776856 10660 tablet_service.cc:1467] Processing CreateTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 (DEFAULT_TABLE table=test-table [id=39cc64e556c14fbaa0e70fe078decb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:38.778156 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b022bddafa6f4e4fb9d90b464901c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:38.777716 10511 tablet_service.cc:1467] Processing CreateTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 (DEFAULT_TABLE table=test-table [id=39cc64e556c14fbaa0e70fe078decb57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:38.778241 10660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b022bddafa6f4e4fb9d90b464901c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:38.778913 10511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b022bddafa6f4e4fb9d90b464901c8d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:38.789021 10903 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Bootstrap starting.
I20260809 22:48:38.792927 10903 tablet_bootstrap.cc:654] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:38.793931 10903 log.cc:824] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:38.796597 10905 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Bootstrap starting.
I20260809 22:48:38.797008 10903 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: No bootstrap required, opened a new log
I20260809 22:48:38.797333 10903 ts_tablet_manager.cc:1397] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:48:38.799351 10905 tablet_bootstrap.cc:654] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:38.799393 10904 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Bootstrap starting.
I20260809 22:48:38.800609 10905 log.cc:824] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:38.803184 10904 tablet_bootstrap.cc:654] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:38.804312 10904 log.cc:824] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:38.805827 10905 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: No bootstrap required, opened a new log
I20260809 22:48:38.805581 10903 raft_consensus.cc:348] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.805994 10903 raft_consensus.cc:374] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:38.806109 10905 ts_tablet_manager.cc:1397] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 22:48:38.806125 10903 raft_consensus.cc:729] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca353b3a6dec4f66a2019a9ee5366873, State: Initialized, Role: FOLLOWER
I20260809 22:48:38.806710 10903 consensus_queue.cc:260] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.807360 10904 tablet_bootstrap.cc:492] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: No bootstrap required, opened a new log
I20260809 22:48:38.807631 10904 ts_tablet_manager.cc:1397] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.003s
I20260809 22:48:38.807849 10880 heartbeater.cc:503] Master 127.21.27.190:45647 was elected leader, sending a full tablet report...
I20260809 22:48:38.809584 10903 ts_tablet_manager.cc:1428] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Time spent starting tablet: real 0.012s user 0.002s sys 0.009s
I20260809 22:48:38.816982 10905 raft_consensus.cc:348] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.817466 10905 raft_consensus.cc:374] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:38.817579 10905 raft_consensus.cc:729] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74d849b9a34466a95c9bb6759e96070, State: Initialized, Role: FOLLOWER
I20260809 22:48:38.818063 10904 raft_consensus.cc:348] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.818248 10905 consensus_queue.cc:260] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.818476 10904 raft_consensus.cc:374] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:38.818567 10904 raft_consensus.cc:729] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32302d316ae7423090fc043bba70849d, State: Initialized, Role: FOLLOWER
I20260809 22:48:38.819204 10904 consensus_queue.cc:260] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.820189 10905 ts_tablet_manager.cc:1428] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260809 22:48:38.820276 10732 heartbeater.cc:503] Master 127.21.27.190:45647 was elected leader, sending a full tablet report...
I20260809 22:48:38.822111 10904 ts_tablet_manager.cc:1428] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260809 22:48:38.889796 10584 tablet.cc:2369] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:48:38.914405 10733 tablet.cc:2369] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:48:38.938117 10881 tablet.cc:2369] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:38.981123 10910 raft_consensus.cc:482] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:38.981458 10910 raft_consensus.cc:504] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.983263 10910 leader_election.cc:288] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca353b3a6dec4f66a2019a9ee5366873 (127.21.27.131:40217), 32302d316ae7423090fc043bba70849d (127.21.27.130:34061)
I20260809 22:48:38.991344 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.984341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:50175 (local address 127.21.27.130:34061)
0809 22:48:38.984521 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.984528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.985406 (+ 878us) server_negotiation.cc:407] Connection header received
0809 22:48:38.985532 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.985536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.985615 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.985716 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:38.986399 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.987444 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.989342 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.990305 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.990461 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.990532 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.990908 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.991045 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:48:38.991472 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.983732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:34061 (local address 127.21.27.129:50175)
0809 22:48:38.985280 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.985296 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.985416 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.985750 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.985755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.985778 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:38.986213 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.986225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.987591 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.987597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.989167 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.989179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.990511 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.990518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.990688 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.990722 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.990852 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.990977 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1436,"mutex_wait_us":53,"thread_start_us":822,"threads_started":1}
I20260809 22:48:38.992398 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.983730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:55821 (local address 127.21.27.131:40217)
0809 22:48:38.983948 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:38.983957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:38.984754 (+ 797us) server_negotiation.cc:407] Connection header received
0809 22:48:38.985140 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:38.985146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:38.985230 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:38.985292 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:38.986160 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.987323 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.989961 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.991200 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.991355 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.991443 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:38.992010 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 22:48:38.992139 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:48:38.992996 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:38.983670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:40217 (local address 127.21.27.129:55821)
0809 22:48:38.984618 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:38.984644 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:38.984941 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:38.985525 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:38.985530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:38.985557 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:38.986003 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.986018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.987463 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.987468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.989770 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:38.989784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:38.991606 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:38.991612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:38.991809 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:38.991843 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:48:38.992590 (+ 747us) client_negotiation.cc:240] Negotiation successful
0809 22:48:38.992739 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":81,"thread_start_us":1286,"threads_started":1}
I20260809 22:48:38.993120 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b022bddafa6f4e4fb9d90b464901c8d5" candidate_uuid: "c74d849b9a34466a95c9bb6759e96070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32302d316ae7423090fc043bba70849d" is_pre_election: true
I20260809 22:48:38.993670 10682 raft_consensus.cc:2454] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c74d849b9a34466a95c9bb6759e96070 in term 0.
I20260809 22:48:38.994046 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b022bddafa6f4e4fb9d90b464901c8d5" candidate_uuid: "c74d849b9a34466a95c9bb6759e96070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca353b3a6dec4f66a2019a9ee5366873" is_pre_election: true
I20260809 22:48:38.994369 10463 leader_election.cc:302] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32302d316ae7423090fc043bba70849d, c74d849b9a34466a95c9bb6759e96070; no voters:
I20260809 22:48:38.994606 10832 raft_consensus.cc:2454] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c74d849b9a34466a95c9bb6759e96070 in term 0.
I20260809 22:48:38.994799 10910 raft_consensus.cc:2781] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:38.994956 10910 raft_consensus.cc:482] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:38.995070 10910 raft_consensus.cc:3037] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:38.997536 10910 raft_consensus.cc:504] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:38.998364 10910 leader_election.cc:288] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [CANDIDATE]: Term 1 election: Requested vote from peers ca353b3a6dec4f66a2019a9ee5366873 (127.21.27.131:40217), 32302d316ae7423090fc043bba70849d (127.21.27.130:34061)
I20260809 22:48:38.998761 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b022bddafa6f4e4fb9d90b464901c8d5" candidate_uuid: "c74d849b9a34466a95c9bb6759e96070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca353b3a6dec4f66a2019a9ee5366873"
I20260809 22:48:38.998880 10682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b022bddafa6f4e4fb9d90b464901c8d5" candidate_uuid: "c74d849b9a34466a95c9bb6759e96070" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32302d316ae7423090fc043bba70849d"
I20260809 22:48:38.998979 10832 raft_consensus.cc:3037] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:38.999099 10682 raft_consensus.cc:3037] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:39.001262 10832 raft_consensus.cc:2454] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c74d849b9a34466a95c9bb6759e96070 in term 1.
I20260809 22:48:39.001379 10682 raft_consensus.cc:2454] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c74d849b9a34466a95c9bb6759e96070 in term 1.
I20260809 22:48:39.001698 10464 leader_election.cc:302] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c74d849b9a34466a95c9bb6759e96070, ca353b3a6dec4f66a2019a9ee5366873; no voters:
I20260809 22:48:39.002046 10910 raft_consensus.cc:2781] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:39.002521 10910 raft_consensus.cc:686] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 1 LEADER]: Becoming Leader. State: Replica: c74d849b9a34466a95c9bb6759e96070, State: Running, Role: LEADER
I20260809 22:48:39.003021 10910 consensus_queue.cc:237] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } }
I20260809 22:48:39.007675 10375 catalog_manager.cc:5495] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 reported cstate change: term changed from 0 to 1, leader changed from <none> to c74d849b9a34466a95c9bb6759e96070 (127.21.27.129). New cstate: current_term: 1 leader_uuid: "c74d849b9a34466a95c9bb6759e96070" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c74d849b9a34466a95c9bb6759e96070" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 39221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 } health_report { overall_health: UNKNOWN } } }
I20260809 22:48:39.068536 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:39.063171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43218 (local address 127.21.27.129:39221)
0809 22:48:39.063439 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:39.063444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:39.063466 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:48:39.063690 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:39.063694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:39.063794 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:39.063919 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:39.064309 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.065024 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.066428 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.066832 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.066928 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:39.067468 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:39.067986 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:39.068092 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:39.068171 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 22:48:39.068236 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 22:48:39.076870 10832 raft_consensus.cc:1261] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 1 FOLLOWER]: Refusing update from remote peer c74d849b9a34466a95c9bb6759e96070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:39.076894 10682 raft_consensus.cc:1261] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 1 FOLLOWER]: Refusing update from remote peer c74d849b9a34466a95c9bb6759e96070: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:39.078017 10914 consensus_queue.cc:1035] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32302d316ae7423090fc043bba70849d" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:39.078454 10910 consensus_queue.cc:1035] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca353b3a6dec4f66a2019a9ee5366873" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 40217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:39.091109 10917 mvcc.cc:205] Tried to move back new op lower bound from 7316749185328123904 to 7316749185040592896. Current Snapshot: MvccSnapshot[applied={T|T < 7316749185328123904}]
I20260809 22:48:39.100574 10918 mvcc.cc:205] Tried to move back new op lower bound from 7316749185328123904 to 7316749185040592896. Current Snapshot: MvccSnapshot[applied={T|T < 7316749185328123904 or (T in {7316749185328123904})}]
I20260809 22:48:39.105892 10375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:39.106191 10375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:39.109836 10375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28b67c4cc5ff4655984137213f36b08d: Sending DeleteTablet for 3 replicas of tablet b022bddafa6f4e4fb9d90b464901c8d5
I20260809 22:48:39.111003 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:39 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:42198
I20260809 22:48:39.111263 10511 tablet_service.cc:1514] Processing DeleteTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:39 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:43204
I20260809 22:48:39.111802 10660 tablet_service.cc:1514] Processing DeleteTablet for tablet b022bddafa6f4e4fb9d90b464901c8d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:39 UTC) from {username='kudu', principal='kudu/127.21.27.190@KRBTEST.COM'} at 127.0.0.1:40508
I20260809 22:48:39.112094 10928 tablet_replica.cc:330] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: stopping tablet replica
I20260809 22:48:39.112660 10928 raft_consensus.cc:2229] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:39.113238 10928 raft_consensus.cc:2258] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:39.114046 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10443
I20260809 22:48:39.115686 10928 ts_tablet_manager.cc:1905] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:39.122098 10930 tablet_replica.cc:330] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: stopping tablet replica
I20260809 22:48:39.122576 10930 raft_consensus.cc:2229] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:39.123119 10930 raft_consensus.cc:2258] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:39.124132 10929 tablet_replica.cc:330] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: stopping tablet replica
I20260809 22:48:39.124569 10929 raft_consensus.cc:2229] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:39.125191 10928 ts_tablet_manager.cc:1918] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:39.125373 10929 raft_consensus.cc:2258] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:39.125524 10928 log.cc:1196] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/b022bddafa6f4e4fb9d90b464901c8d5
I20260809 22:48:39.126187 10928 ts_tablet_manager.cc:1939] T b022bddafa6f4e4fb9d90b464901c8d5 P ca353b3a6dec4f66a2019a9ee5366873: Deleting consensus metadata
I20260809 22:48:39.128281 10361 catalog_manager.cc:4841] TS ca353b3a6dec4f66a2019a9ee5366873 (127.21.27.131:40217): tablet b022bddafa6f4e4fb9d90b464901c8d5 (table test-table [id=39cc64e556c14fbaa0e70fe078decb57]) successfully deleted
I20260809 22:48:39.128508 10929 ts_tablet_manager.cc:1905] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:39.130811 10930 ts_tablet_manager.cc:1905] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:39.136581 10929 ts_tablet_manager.cc:1918] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:39.136792 10929 log.cc:1196] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/b022bddafa6f4e4fb9d90b464901c8d5
I20260809 22:48:39.137399 10929 ts_tablet_manager.cc:1939] T b022bddafa6f4e4fb9d90b464901c8d5 P c74d849b9a34466a95c9bb6759e96070: Deleting consensus metadata
I20260809 22:48:39.138264 10930 ts_tablet_manager.cc:1918] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:39.138429 10930 log.cc:1196] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/b022bddafa6f4e4fb9d90b464901c8d5
Aug 09 22:48:39 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.129@KRBTEST.COM
I20260809 22:48:39.138978 10930 ts_tablet_manager.cc:1939] T b022bddafa6f4e4fb9d90b464901c8d5 P 32302d316ae7423090fc043bba70849d: Deleting consensus metadata
I20260809 22:48:39.139487 10361 catalog_manager.cc:4841] TS c74d849b9a34466a95c9bb6759e96070 (127.21.27.129:39221): tablet b022bddafa6f4e4fb9d90b464901c8d5 (table test-table [id=39cc64e556c14fbaa0e70fe078decb57]) successfully deleted
I20260809 22:48:39.140749 10360 catalog_manager.cc:4841] TS 32302d316ae7423090fc043bba70849d (127.21.27.130:34061): tablet b022bddafa6f4e4fb9d90b464901c8d5 (table test-table [id=39cc64e556c14fbaa0e70fe078decb57]) successfully deleted
I20260809 22:48:39.146823 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:39.115558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.21.27.129:39221)
0809 22:48:39.115752 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:39.115760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:39.126243 (+ 10483us) server_negotiation.cc:407] Connection header received
0809 22:48:39.126785 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:39.126790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:39.126875 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:39.126986 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:39.129439 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.130743 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.132859 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.133627 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.133738 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:39.140218 (+ 6480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:39.140230 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:39.140244 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:39.140288 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:39.143855 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.144607 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.144615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.144617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.144778 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.145214 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.145218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.145221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.145580 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:39.145891 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:39.146244 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:48:39.146496 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:48:39.147621 10578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:39.247750 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10443
I20260809 22:48:39.269887 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10590
Aug 09 22:48:39 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.130@KRBTEST.COM
I20260809 22:48:39.285241 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:39.270716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.21.27.130:34061)
0809 22:48:39.270883 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:39.270888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:39.270906 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:48:39.271007 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:39.271011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:39.271073 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:39.271224 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:39.272492 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.273249 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.274634 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.275180 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.275289 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:39.279591 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:39.279608 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:39.279633 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:39.279669 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:39.282524 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.283282 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.283294 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.283300 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.283458 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.283839 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.283842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.283844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.284209 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:39.284433 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:39.284689 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 22:48:39.284855 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:48:39.285965 10727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:39.377231 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10590
I20260809 22:48:39.397837 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10739
Aug 09 22:48:39 dist-test-slave-chsk krb5kdc[10329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.131@KRBTEST.COM
I20260809 22:48:39.413115 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:39.398698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.21.27.131:40217)
0809 22:48:39.398893 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:39.398902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:39.398925 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:48:39.399204 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:39.399208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:39.399288 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:39.399411 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:39.400785 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.401529 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.402858 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:39.403388 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:39.403543 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:39.407996 (+ 4453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:39.408011 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:39.408027 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:39.408059 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:39.410591 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.411286 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.411293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.411295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.411433 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:39.411760 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:39.411763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:39.411765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:39.412101 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:39.412336 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:39.412585 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 22:48:39.412744 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:48:39.413772 10875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:39.502503 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10739
I20260809 22:48:39.522317 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10342
I20260809 22:48:39.523543 10411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:39.634047 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10342
2026-08-09T22:48:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4602 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T22:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:39Z Disabled control of system clock
I20260809 22:48:39.721300 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45913
--webserver_interface=127.21.27.190
--webserver_port=0
--builtin_ntp_servers=127.21.27.148:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:40.056598 10940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:40.056882 10940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:40.056965 10940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:40.064189 10940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:40.064293 10940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:40.064332 10940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:40.064388 10940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:40.064432 10940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:40.073534 10940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45913
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:40.074832 10940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:40.076561 10940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:40.088418 10945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:40.088375 10947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:40.088326 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:40.089594 10940 server_base.cc:1034] running on GCE node
I20260809 22:48:40.090410 10940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:40.091912 10940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:40.093173 10940 hybrid_clock.cc:648] HybridClock initialized: now 1786315720093125 us; error 62 us; skew 500 ppm
I20260809 22:48:40.093538 10940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:40.095844 10940 webserver.cc:460] Webserver started at http://127.21.27.190:45431/ using document root <none> and password file <none>
I20260809 22:48:40.096531 10940 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:40.096666 10940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:40.097008 10940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:40.100054 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "226be94ad0e5444fa8fd8dfefb88eb95"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "19bf225a646122fa6630e5780278be79"
server_key_iv: "7015dca76058113ff97e0c1b19cdfb21"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.100849 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "226be94ad0e5444fa8fd8dfefb88eb95"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "19bf225a646122fa6630e5780278be79"
server_key_iv: "7015dca76058113ff97e0c1b19cdfb21"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.105980 10940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:48:40.109436 10956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:40.110725 10940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:48:40.110924 10940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "226be94ad0e5444fa8fd8dfefb88eb95"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "19bf225a646122fa6630e5780278be79"
server_key_iv: "7015dca76058113ff97e0c1b19cdfb21"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.111182 10940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:40.121866 10940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:40.122673 10940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:40.122973 10940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:40.141776 10940 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45913
I20260809 22:48:40.141795 11010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45913 every 8 connection(s)
I20260809 22:48:40.143186 10940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:40.147662 11011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:40.150796 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 10940
I20260809 22:48:40.150985 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:40.151503 21614 external_mini_cluster.cc:1428] Setting key 339508704e4b08d04c1acf5228529453
I20260809 22:48:40.160132 11011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: Bootstrap starting.
I20260809 22:48:40.163880 11011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:40.164968 11011 log.cc:824] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:40.165347 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:40.153451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.21.27.190:45913)
0809 22:48:40.154469 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:40.154479 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:40.154521 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:48:40.156186 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:40.156208 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:40.156672 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:40.157001 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:40.157634 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.158910 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.160779 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.161442 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.161585 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:40.162855 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:40.162862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:40.162874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:40.163484 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:40.163578 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:40.163599 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:40.163691 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:40.163945 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:48:40.164315 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":375,"threads_started":1}
I20260809 22:48:40.167774 11011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: No bootstrap required, opened a new log
I20260809 22:48:40.175501 11011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } }
I20260809 22:48:40.175927 11011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:40.176039 11011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226be94ad0e5444fa8fd8dfefb88eb95, State: Initialized, Role: FOLLOWER
I20260809 22:48:40.176604 11011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } }
I20260809 22:48:40.176872 11011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:40.177021 11011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:40.177201 11011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:40.179580 11011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } }
I20260809 22:48:40.180109 11011 leader_election.cc:302] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 226be94ad0e5444fa8fd8dfefb88eb95; no voters:
I20260809 22:48:40.180895 11011 leader_election.cc:288] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:40.181023 11016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:40.181859 11016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [term 1 LEADER]: Becoming Leader. State: Replica: 226be94ad0e5444fa8fd8dfefb88eb95, State: Running, Role: LEADER
I20260809 22:48:40.182416 11016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } }
I20260809 22:48:40.183055 11011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:40.185624 11018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 226be94ad0e5444fa8fd8dfefb88eb95. Latest consensus state: current_term: 1 leader_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } } }
I20260809 22:48:40.185623 11017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "226be94ad0e5444fa8fd8dfefb88eb95" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45913 } } }
I20260809 22:48:40.186177 11018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:40.186205 11017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:40.188807 11026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:40.193923 11026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:40.203228 11026 catalog_manager.cc:1347] Generated new cluster ID: 7ce12f5ddb17456c9b5bbf2ecaecec6c
I20260809 22:48:40.203351 11026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:40.233970 11026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:40.235366 11026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:40.243817 11026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: Generated new TSK 0
I20260809 22:48:40.244745 11026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:40.262423 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.129:0
--local_ip_for_outbound_sockets=127.21.27.129
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--builtin_ntp_servers=127.21.27.148:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 22:48:40.617846 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:40.618103 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:40.618187 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:40.625551 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:40.625674 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:40.625836 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.129
I20260809 22:48:40.635460 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.27.129
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.129
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:40.636699 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:40.638370 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:40.651088 11046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:40.651201 11047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:40.651219 11050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:40.651849 11039 server_base.cc:1034] running on GCE node
I20260809 22:48:40.652508 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:40.653457 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:40.654656 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786315720654619 us; error 36 us; skew 500 ppm
I20260809 22:48:40.654958 11039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:40.657207 11039 webserver.cc:460] Webserver started at http://127.21.27.129:34405/ using document root <none> and password file <none>
I20260809 22:48:40.657864 11039 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:40.657979 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:40.658313 11039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:40.661126 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data/instance:
uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "161535606a8cede34d4b4a8527cc7ac9"
server_key_iv: "49fb70c11e9902fca4fe7620130c06b1"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.661906 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance:
uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "161535606a8cede34d4b4a8527cc7ac9"
server_key_iv: "49fb70c11e9902fca4fe7620130c06b1"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.667069 11039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 22:48:40.670486 11056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:40.671792 11039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:48:40.671967 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal
uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93"
format_stamp: "Formatted at 2026-08-09 22:48:40 on dist-test-slave-chsk"
server_key: "161535606a8cede34d4b4a8527cc7ac9"
server_key_iv: "49fb70c11e9902fca4fe7620130c06b1"
server_key_version: "encryptionkey@0"
I20260809 22:48:40.672173 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:40.692270 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:40.693063 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:40.693383 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:40.694334 11039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:40.695832 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:40.695942 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:40.696072 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:40.696177 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:40.729765 11039 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.129:37189
I20260809 22:48:40.729885 11179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.129:37189 every 8 connection(s)
I20260809 22:48:40.731032 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/data/info.pb
I20260809 22:48:40.737787 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11039
I20260809 22:48:40.737977 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal/instance
I20260809 22:48:40.738581 21614 external_mini_cluster.cc:1428] Setting key 3c3f1f4a40a6c7c9676160af0de650e3
I20260809 22:48:40.741570 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.130:0
--local_ip_for_outbound_sockets=127.21.27.130
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--builtin_ntp_servers=127.21.27.148:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:40.744380 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:40.733543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.129:33917 (local address 127.21.27.190:45913)
0809 22:48:40.733980 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:40.733985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:40.735124 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 22:48:40.736891 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:40.736895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:40.736958 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:40.737052 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:40.738414 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.739268 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.741304 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.741983 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.742151 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:40.743559 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:40.743564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:40.743567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:40.743586 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:40.743631 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:40.743644 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:40.743719 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:40.743993 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 22:48:40.744132 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":205,"threads_started":1}
I20260809 22:48:40.745652 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:40.734114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45913 (local address 127.21.27.129:33917)
0809 22:48:40.734966 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:40.735011 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:40.736658 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:40.737255 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:40.737262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:40.737372 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:40.738218 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:40.738244 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.739454 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.739458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:40.741151 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:40.741163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:40.742197 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:40.742202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:40.742396 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:40.743172 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:40.743179 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:40.743230 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:40.743338 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:40.743749 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:40.743763 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:40.743773 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:48:40.743950 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 22:48:40.744351 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":279,"threads_started":1}
I20260809 22:48:40.747254 11180 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45913
I20260809 22:48:40.747573 11180 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:40.748297 11180 heartbeater.cc:511] Master 127.21.27.190:45913 requested a full tablet report, sending...
I20260809 22:48:40.750332 10974 ts_manager.cc:194] Registered new tserver with Master: c2b7a84fbd6447b2be3fa6c6bdb15e93 (127.21.27.129:37189)
I20260809 22:48:40.751819 10974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.129:33917
W20260809 22:48:41.095497 11184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:41.095772 11184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:41.095928 11184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:41.103214 11184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:41.103338 11184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:41.103523 11184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.130
I20260809 22:48:41.112684 11184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.27.130
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.130
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:41.113962 11184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:41.116007 11184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:41.129258 11195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:41.129351 11191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:41.129318 11192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:41.130584 11184 server_base.cc:1034] running on GCE node
I20260809 22:48:41.131323 11184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:41.132349 11184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:41.133598 11184 hybrid_clock.cc:648] HybridClock initialized: now 1786315721133549 us; error 43 us; skew 500 ppm
I20260809 22:48:41.133924 11184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:41.136260 11184 webserver.cc:460] Webserver started at http://127.21.27.130:45527/ using document root <none> and password file <none>
I20260809 22:48:41.136953 11184 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:41.137100 11184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:41.137425 11184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:41.140340 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data/instance:
uuid: "7c1f1c5dad284e138c08e93f415ca572"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "2a10c609f804bc67f28c0a216aa7a4bd"
server_key_iv: "7b464affd970d020affa87616ff7c181"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.141132 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance:
uuid: "7c1f1c5dad284e138c08e93f415ca572"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "2a10c609f804bc67f28c0a216aa7a4bd"
server_key_iv: "7b464affd970d020affa87616ff7c181"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.146297 11184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:48:41.149765 11201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:41.151111 11184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:48:41.151335 11184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal
uuid: "7c1f1c5dad284e138c08e93f415ca572"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "2a10c609f804bc67f28c0a216aa7a4bd"
server_key_iv: "7b464affd970d020affa87616ff7c181"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.151543 11184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:41.167366 11184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:41.168156 11184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:41.168467 11184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:41.169423 11184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:41.170827 11184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:41.170955 11184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:41.171098 11184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:41.171214 11184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 22:48:41.205175 11184 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.130:36415
I20260809 22:48:41.205231 11322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.130:36415 every 8 connection(s)
I20260809 22:48:41.206530 11184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/data/info.pb
I20260809 22:48:41.213549 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11184
I20260809 22:48:41.213835 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal/instance
I20260809 22:48:41.214280 21614 external_mini_cluster.cc:1428] Setting key 003aec23d22e964dd8a6200b408d8e97
I20260809 22:48:41.217336 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.27.131:0
--local_ip_for_outbound_sockets=127.21.27.131
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--builtin_ntp_servers=127.21.27.148:36493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 22:48:41.219247 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.209165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.130:59545 (local address 127.21.27.190:45913)
0809 22:48:41.209350 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.209354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.210606 (+ 1252us) server_negotiation.cc:407] Connection header received
0809 22:48:41.212367 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.212371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.212436 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.212525 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:41.213786 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.214659 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.216428 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.216813 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.216910 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.218485 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:41.218490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:41.218493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:41.218510 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:41.218540 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:41.218559 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:41.218625 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.218879 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.218955 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:48:41.220639 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.209616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45913 (local address 127.21.27.130:59545)
0809 22:48:41.210462 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:41.210496 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:41.212131 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:41.212709 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:41.212716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:41.212791 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:41.213507 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.213534 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.214865 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.214868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.216268 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.216285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.217047 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.217057 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.217388 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.218123 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:41.218128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:41.218189 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:41.218289 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:41.218657 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:41.218671 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:41.218681 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:48:41.218863 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 22:48:41.219308 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":265,"threads_started":1}
I20260809 22:48:41.222028 11323 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45913
I20260809 22:48:41.222335 11323 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:41.223067 11323 heartbeater.cc:511] Master 127.21.27.190:45913 requested a full tablet report, sending...
I20260809 22:48:41.224496 10974 ts_manager.cc:194] Registered new tserver with Master: 7c1f1c5dad284e138c08e93f415ca572 (127.21.27.130:36415)
I20260809 22:48:41.225288 10974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.130:59545
W20260809 22:48:41.552306 11326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:41.552590 11326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:41.552678 11326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:41.560104 11326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:41.560204 11326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:41.560310 11326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.27.131
I20260809 22:48:41.569118 11326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:36493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.27.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.27.131
--webserver_port=0
--tserver_master_addrs=127.21.27.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.27.131
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:41.570310 11326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:41.571976 11326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:41.584829 11336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:41.584920 11332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:41.584862 11333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:41.585727 11326 server_base.cc:1034] running on GCE node
I20260809 22:48:41.586407 11326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:41.587423 11326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:41.588649 11326 hybrid_clock.cc:648] HybridClock initialized: now 1786315721588613 us; error 51 us; skew 500 ppm
I20260809 22:48:41.588985 11326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:48:41.591423 11326 webserver.cc:460] Webserver started at http://127.21.27.131:34883/ using document root <none> and password file <none>
I20260809 22:48:41.592103 11326 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:41.592247 11326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:41.592566 11326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:41.595439 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data/instance:
uuid: "a1f79089f984489aad5a6e5bb9e20293"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "ff7c566414f7be1085c3533ae7eb6f4c"
server_key_iv: "7d438847956ce1b36f3a1f4293e575f5"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.596236 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance:
uuid: "a1f79089f984489aad5a6e5bb9e20293"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "ff7c566414f7be1085c3533ae7eb6f4c"
server_key_iv: "7d438847956ce1b36f3a1f4293e575f5"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.601336 11326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:48:41.604739 11343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:41.605996 11326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:48:41.606182 11326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal
uuid: "a1f79089f984489aad5a6e5bb9e20293"
format_stamp: "Formatted at 2026-08-09 22:48:41 on dist-test-slave-chsk"
server_key: "ff7c566414f7be1085c3533ae7eb6f4c"
server_key_iv: "7d438847956ce1b36f3a1f4293e575f5"
server_key_version: "encryptionkey@0"
I20260809 22:48:41.606403 11326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:41.618400 11326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:41.619235 11326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:41.619530 11326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:41.620479 11326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:48:41.621846 11326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:48:41.621965 11326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:41.622133 11326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:48:41.622231 11326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:41.659008 11326 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.131:45257
I20260809 22:48:41.659101 11464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.131:45257 every 8 connection(s)
I20260809 22:48:41.660356 11326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/data/info.pb
I20260809 22:48:41.668406 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11326
I20260809 22:48:41.668598 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal/instance
I20260809 22:48:41.669045 21614 external_mini_cluster.cc:1428] Setting key d5567c4e3edd943aafe97910cdc14566
I20260809 22:48:41.673072 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.662855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:50163 (local address 127.21.27.190:45913)
0809 22:48:41.663036 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.663040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.664466 (+ 1426us) server_negotiation.cc:407] Connection header received
0809 22:48:41.666253 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.666259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.666356 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.666476 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:41.667585 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.668377 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.670224 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.670768 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.670881 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.672399 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:41.672403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:41.672406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:41.672422 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:41.672454 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:41.672465 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:41.672536 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.672757 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.672833 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:48:41.673986 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.663362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:45913 (local address 127.21.27.131:50163)
0809 22:48:41.664298 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:41.664340 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:41.665971 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:41.666705 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:41.666712 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:41.666794 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:48:41.667405 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.667423 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.668539 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.668546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.670041 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.670076 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.670998 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.671006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.671297 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.672109 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:48:41.672112 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:48:41.672160 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:48:41.672229 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:48:41.672549 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:48:41.672558 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:48:41.672563 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:48:41.672707 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 22:48:41.672981 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":297,"threads_started":1}
I20260809 22:48:41.675414 11465 heartbeater.cc:347] Connected to a master server at 127.21.27.190:45913
I20260809 22:48:41.675721 11465 heartbeater.cc:464] Registering TS with master...
I20260809 22:48:41.676532 11465 heartbeater.cc:511] Master 127.21.27.190:45913 requested a full tablet report, sending...
I20260809 22:48:41.677883 10974 ts_manager.cc:194] Registered new tserver with Master: a1f79089f984489aad5a6e5bb9e20293 (127.21.27.131:45257)
I20260809 22:48:41.678687 10974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.27.131:50163
I20260809 22:48:41.683987 21614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:48:41.693825 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.688313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.21.27.190:45913)
0809 22:48:41.688496 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.688500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.688982 (+ 482us) server_negotiation.cc:407] Connection header received
0809 22:48:41.689190 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.689194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.689265 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.689344 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:41.689769 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.690504 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.691869 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.692254 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.692349 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.693275 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:41.693280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:41.693283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:41.693299 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:41.693330 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:41.693341 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:41.693408 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.693531 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.693601 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:48:41.697314 10974 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:48:41.704133 10974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:48:41.727313 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.715678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:36415 (local address 127.0.0.1:47212)
0809 22:48:41.717436 (+ 1758us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:41.717485 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:41.717705 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:41.719254 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:41.719260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:41.719290 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:41.719730 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.719744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.721755 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.721760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.724190 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.724206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.726383 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.726389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.726602 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.726650 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:48:41.726790 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 22:48:41.726964 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":2675,"threads_started":1}
I20260809 22:48:41.728430 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.716132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.131:45257 (local address 127.0.0.1:52832)
0809 22:48:41.718597 (+ 2465us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:41.718613 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:41.718720 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:41.720408 (+ 1688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:41.720413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:41.720438 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:41.720804 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.720816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.723290 (+ 2474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.723295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.725473 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.725488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.727657 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.727664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.727877 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.727914 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 22:48:41.727971 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 22:48:41.728122 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2343,"thread_start_us":1105,"threads_started":1}
I20260809 22:48:41.728592 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.715904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.21.27.130:36415)
0809 22:48:41.718307 (+ 2403us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.718318 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.718350 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:48:41.718451 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.718458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.718721 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.718925 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:41.719957 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.721568 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.724434 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.725872 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.726006 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.726256 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.727974 (+ 1718us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.728265 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2235,"thread_start_us":2119,"threads_started":1}
I20260809 22:48:41.732574 11253 tablet_service.cc:1467] Processing CreateTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 (DEFAULT_TABLE table=test-table [id=210371d78b654396958e2e029b61e2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:41.734133 11253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b535a0b0c641e38d1725762c1d53f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:41.743683 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.716872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.21.27.131:45257)
0809 22:48:41.719806 (+ 2934us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.719814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.719846 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:48:41.719943 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.719949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.720180 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.721308 (+ 1128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:41.721568 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.723089 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.726026 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.727430 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.728989 (+ 1559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.729376 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.729445 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.743297 (+ 13852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2754,"thread_start_us":3085,"threads_started":1}
I20260809 22:48:41.747486 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 (DEFAULT_TABLE table=test-table [id=210371d78b654396958e2e029b61e2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:41.749069 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b535a0b0c641e38d1725762c1d53f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:41.749220 11487 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Bootstrap starting.
I20260809 22:48:41.750437 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.719843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.21.27.129:37189)
0809 22:48:41.742034 (+ 22191us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:41.742043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:41.742075 (+ 32us) server_negotiation.cc:407] Connection header received
0809 22:48:41.742164 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:41.742172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:41.742374 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:41.742566 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:41.743226 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.744483 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.746751 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.748601 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.748741 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.748995 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:41.749919 (+ 924us) server_negotiation.cc:299] Negotiation successful
0809 22:48:41.750082 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21971,"thread_start_us":22895,"threads_started":1}
I20260809 22:48:41.750586 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:41.719586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37189 (local address 127.0.0.1:51346)
0809 22:48:41.727704 (+ 8118us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:41.727722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:41.727821 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:41.742636 (+ 14815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:41.742645 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:41.742693 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:41.742969 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.742984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.744644 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.744652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.746536 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:41.746564 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:41.749106 (+ 2542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:41.749113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:41.749319 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:41.749353 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:48:41.749450 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:48:41.750279 (+ 829us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7876,"thread_start_us":23428,"threads_started":1}
I20260809 22:48:41.754154 11110 tablet_service.cc:1467] Processing CreateTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 (DEFAULT_TABLE table=test-table [id=210371d78b654396958e2e029b61e2a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:48:41.755239 11180 heartbeater.cc:503] Master 127.21.27.190:45913 was elected leader, sending a full tablet report...
I20260809 22:48:41.755473 11110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98b535a0b0c641e38d1725762c1d53f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:41.760686 11487 tablet_bootstrap.cc:654] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:41.761883 11488 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Bootstrap starting.
I20260809 22:48:41.762055 11487 log.cc:824] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:41.765132 11487 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: No bootstrap required, opened a new log
I20260809 22:48:41.765401 11487 ts_tablet_manager.cc:1397] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260809 22:48:41.765398 11488 tablet_bootstrap.cc:654] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:41.766551 11488 log.cc:824] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:41.768189 11491 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Bootstrap starting.
I20260809 22:48:41.769543 11488 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: No bootstrap required, opened a new log
I20260809 22:48:41.769791 11488 ts_tablet_manager.cc:1397] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 22:48:41.771556 11491 tablet_bootstrap.cc:654] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:41.772392 11491 log.cc:824] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:41.774577 11491 tablet_bootstrap.cc:492] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: No bootstrap required, opened a new log
I20260809 22:48:41.774825 11491 ts_tablet_manager.cc:1397] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:48:41.775285 11487 raft_consensus.cc:348] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.775683 11487 raft_consensus.cc:374] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:41.775776 11487 raft_consensus.cc:729] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1f1c5dad284e138c08e93f415ca572, State: Initialized, Role: FOLLOWER
I20260809 22:48:41.776371 11487 consensus_queue.cc:260] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.780977 11488 raft_consensus.cc:348] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.781508 11488 raft_consensus.cc:374] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:41.781651 11488 raft_consensus.cc:729] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f79089f984489aad5a6e5bb9e20293, State: Initialized, Role: FOLLOWER
I20260809 22:48:41.782377 11488 consensus_queue.cc:260] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.782943 11491 raft_consensus.cc:348] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.783496 11491 raft_consensus.cc:374] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:41.783610 11491 raft_consensus.cc:729] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2b7a84fbd6447b2be3fa6c6bdb15e93, State: Initialized, Role: FOLLOWER
I20260809 22:48:41.783687 11465 heartbeater.cc:503] Master 127.21.27.190:45913 was elected leader, sending a full tablet report...
I20260809 22:48:41.784158 11491 consensus_queue.cc:260] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:41.784276 11488 ts_tablet_manager.cc:1428] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 22:48:41.785398 11487 ts_tablet_manager.cc:1428] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
I20260809 22:48:41.785701 11491 ts_tablet_manager.cc:1428] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 22:48:41.785861 11323 heartbeater.cc:503] Master 127.21.27.190:45913 was elected leader, sending a full tablet report...
W20260809 22:48:41.912091 11466 tablet.cc:2369] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:48:41.958739 11324 tablet.cc:2369] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:48:41.989637 11181 tablet.cc:2369] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:48:42.064030 11494 raft_consensus.cc:482] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:48:42.064401 11494 raft_consensus.cc:504] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:42.066211 11494 leader_election.cc:288] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c1f1c5dad284e138c08e93f415ca572 (127.21.27.130:36415), c2b7a84fbd6447b2be3fa6c6bdb15e93 (127.21.27.129:37189)
I20260809 22:48:42.072894 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.066867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:59247 (local address 127.21.27.130:36415)
0809 22:48:42.067187 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.067196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.067224 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:48:42.067316 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.067321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.067403 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.067502 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:42.068040 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.069049 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.070970 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.071973 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.072099 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.072149 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.072496 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.072599 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:48:42.072959 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.066523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.130:36415 (local address 127.21.27.131:59247)
0809 22:48:42.066742 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:42.066765 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:42.066908 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:42.067575 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:42.067579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:42.067610 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:42.067870 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:42.067882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.069186 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.069192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:42.070795 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:42.070808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.072139 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.072145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:42.072325 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.072363 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:48:42.072468 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 22:48:42.072627 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:48:42.073611 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.066834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.27.131:43491 (local address 127.21.27.129:37189)
0809 22:48:42.067052 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.067059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.067744 (+ 685us) server_negotiation.cc:407] Connection header received
0809 22:48:42.067873 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.067878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.067981 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.068079 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:48:42.068710 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.069591 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.071542 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.072578 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.072726 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.072802 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.073207 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.073352 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:48:42.074126 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.066637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.129:37189 (local address 127.21.27.131:43491)
0809 22:48:42.067620 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0809 22:48:42.067635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:48:42.067745 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:48:42.068124 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:48:42.068130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:48:42.068158 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:48:42.068526 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:42.068538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.069742 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.069745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:42.071340 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:48:42.071354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.072751 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.072758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:48:42.072952 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.072987 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:48:42.073805 (+ 818us) client_negotiation.cc:240] Negotiation successful
0809 22:48:42.073917 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":637,"threads_started":1}
I20260809 22:48:42.074131 11275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b535a0b0c641e38d1725762c1d53f7" candidate_uuid: "a1f79089f984489aad5a6e5bb9e20293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1f1c5dad284e138c08e93f415ca572" is_pre_election: true
I20260809 22:48:42.074738 11275 raft_consensus.cc:2454] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1f79089f984489aad5a6e5bb9e20293 in term 0.
I20260809 22:48:42.075191 11132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b535a0b0c641e38d1725762c1d53f7" candidate_uuid: "a1f79089f984489aad5a6e5bb9e20293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" is_pre_election: true
I20260809 22:48:42.075480 11347 leader_election.cc:302] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c1f1c5dad284e138c08e93f415ca572, a1f79089f984489aad5a6e5bb9e20293; no voters:
I20260809 22:48:42.075675 11132 raft_consensus.cc:2454] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1f79089f984489aad5a6e5bb9e20293 in term 0.
I20260809 22:48:42.075924 11494 raft_consensus.cc:2781] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:48:42.076092 11494 raft_consensus.cc:482] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:48:42.076208 11494 raft_consensus.cc:3037] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:42.078467 11494 raft_consensus.cc:504] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:42.079098 11494 leader_election.cc:288] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [CANDIDATE]: Term 1 election: Requested vote from peers 7c1f1c5dad284e138c08e93f415ca572 (127.21.27.130:36415), c2b7a84fbd6447b2be3fa6c6bdb15e93 (127.21.27.129:37189)
I20260809 22:48:42.079594 11132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b535a0b0c641e38d1725762c1d53f7" candidate_uuid: "a1f79089f984489aad5a6e5bb9e20293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93"
I20260809 22:48:42.079594 11275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98b535a0b0c641e38d1725762c1d53f7" candidate_uuid: "a1f79089f984489aad5a6e5bb9e20293" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1f1c5dad284e138c08e93f415ca572"
I20260809 22:48:42.079788 11132 raft_consensus.cc:3037] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:42.079810 11275 raft_consensus.cc:3037] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:42.081988 11275 raft_consensus.cc:2454] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1f79089f984489aad5a6e5bb9e20293 in term 1.
I20260809 22:48:42.081990 11132 raft_consensus.cc:2454] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1f79089f984489aad5a6e5bb9e20293 in term 1.
I20260809 22:48:42.082458 11345 leader_election.cc:302] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1f79089f984489aad5a6e5bb9e20293, c2b7a84fbd6447b2be3fa6c6bdb15e93; no voters:
I20260809 22:48:42.082811 11494 raft_consensus.cc:2781] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:42.083328 11494 raft_consensus.cc:686] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 1 LEADER]: Becoming Leader. State: Replica: a1f79089f984489aad5a6e5bb9e20293, State: Running, Role: LEADER
I20260809 22:48:42.083803 11494 consensus_queue.cc:237] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } }
I20260809 22:48:42.088822 10974 catalog_manager.cc:5495] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1f79089f984489aad5a6e5bb9e20293 (127.21.27.131). New cstate: current_term: 1 leader_uuid: "a1f79089f984489aad5a6e5bb9e20293" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1f79089f984489aad5a6e5bb9e20293" member_type: VOTER last_known_addr { host: "127.21.27.131" port: 45257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 } health_report { overall_health: UNKNOWN } } }
I20260809 22:48:42.126497 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.121260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.21.27.131:45257)
0809 22:48:42.121429 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.121434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.121453 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:48:42.121649 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.121653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.121733 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.121863 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:48:42.122198 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.122928 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.124464 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.124935 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.125064 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.125518 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:48:42.126048 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:48:42.126170 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.126240 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.126302 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:48:42.134690 11275 raft_consensus.cc:1261] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 1 FOLLOWER]: Refusing update from remote peer a1f79089f984489aad5a6e5bb9e20293: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:42.134779 11132 raft_consensus.cc:1261] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 1 FOLLOWER]: Refusing update from remote peer a1f79089f984489aad5a6e5bb9e20293: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:48:42.135747 11498 consensus_queue.cc:1035] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2b7a84fbd6447b2be3fa6c6bdb15e93" member_type: VOTER last_known_addr { host: "127.21.27.129" port: 37189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:42.136130 11494 consensus_queue.cc:1035] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1f1c5dad284e138c08e93f415ca572" member_type: VOTER last_known_addr { host: "127.21.27.130" port: 36415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:48:42.150260 11502 mvcc.cc:205] Tried to move back new op lower bound from 7316749197853360128 to 7316749197659471872. Current Snapshot: MvccSnapshot[applied={T|T < 7316749197853360128}]
I20260809 22:48:42.150588 11501 mvcc.cc:205] Tried to move back new op lower bound from 7316749197853360128 to 7316749197659471872. Current Snapshot: MvccSnapshot[applied={T|T < 7316749197853360128}]
I20260809 22:48:42.167994 10974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:42.168282 10974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:48:42.172147 10974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 226be94ad0e5444fa8fd8dfefb88eb95: Sending DeleteTablet for 3 replicas of tablet 98b535a0b0c641e38d1725762c1d53f7
I20260809 22:48:42.173333 11253 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:42 UTC) from {username='slave'} at 127.0.0.1:47212
I20260809 22:48:42.173468 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:42 UTC) from {username='slave'} at 127.0.0.1:52832
I20260809 22:48:42.174397 11511 tablet_replica.cc:330] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: stopping tablet replica
I20260809 22:48:42.174456 11110 tablet_service.cc:1514] Processing DeleteTablet for tablet 98b535a0b0c641e38d1725762c1d53f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:48:42 UTC) from {username='slave'} at 127.0.0.1:51346
I20260809 22:48:42.174984 11511 raft_consensus.cc:2229] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:42.175657 11511 raft_consensus.cc:2258] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:42.176652 11512 tablet_replica.cc:330] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: stopping tablet replica
I20260809 22:48:42.177235 11512 raft_consensus.cc:2229] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:48:42.177390 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11039
I20260809 22:48:42.178130 11512 raft_consensus.cc:2258] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:42.181375 11512 ts_tablet_manager.cc:1905] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:42.185928 11511 ts_tablet_manager.cc:1905] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:42.190126 11513 tablet_replica.cc:330] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: stopping tablet replica
I20260809 22:48:42.190589 11513 raft_consensus.cc:2229] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:48:42.191128 11513 raft_consensus.cc:2258] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:48:42.191480 11512 ts_tablet_manager.cc:1918] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:42.192392 11512 log.cc:1196] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-2/wal/wals/98b535a0b0c641e38d1725762c1d53f7
I20260809 22:48:42.193078 11512 ts_tablet_manager.cc:1939] T 98b535a0b0c641e38d1725762c1d53f7 P a1f79089f984489aad5a6e5bb9e20293: Deleting consensus metadata
I20260809 22:48:42.193228 11513 ts_tablet_manager.cc:1905] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:48:42.195272 10959 catalog_manager.cc:4841] TS a1f79089f984489aad5a6e5bb9e20293 (127.21.27.131:45257): tablet 98b535a0b0c641e38d1725762c1d53f7 (table test-table [id=210371d78b654396958e2e029b61e2a7]) successfully deleted
I20260809 22:48:42.197098 11511 ts_tablet_manager.cc:1918] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:42.197274 11511 log.cc:1196] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-1/wal/wals/98b535a0b0c641e38d1725762c1d53f7
I20260809 22:48:42.197861 11511 ts_tablet_manager.cc:1939] T 98b535a0b0c641e38d1725762c1d53f7 P 7c1f1c5dad284e138c08e93f415ca572: Deleting consensus metadata
I20260809 22:48:42.199016 11513 ts_tablet_manager.cc:1918] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:48:42.199206 11513 log.cc:1196] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Deleting WAL directory at /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786315394803050-21614-0/minicluster-data/ts-0/wal/wals/98b535a0b0c641e38d1725762c1d53f7
I20260809 22:48:42.199715 11513 ts_tablet_manager.cc:1939] T 98b535a0b0c641e38d1725762c1d53f7 P c2b7a84fbd6447b2be3fa6c6bdb15e93: Deleting consensus metadata
I20260809 22:48:42.201135 10958 catalog_manager.cc:4841] TS c2b7a84fbd6447b2be3fa6c6bdb15e93 (127.21.27.129:37189): tablet 98b535a0b0c641e38d1725762c1d53f7 (table test-table [id=210371d78b654396958e2e029b61e2a7]) successfully deleted
I20260809 22:48:42.202423 10960 catalog_manager.cc:4841] TS 7c1f1c5dad284e138c08e93f415ca572 (127.21.27.130:36415): tablet 98b535a0b0c641e38d1725762c1d53f7 (table test-table [id=210371d78b654396958e2e029b61e2a7]) successfully deleted
I20260809 22:48:42.212191 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.180100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.21.27.129:37189)
0809 22:48:42.180361 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.180368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.200871 (+ 20503us) server_negotiation.cc:407] Connection header received
0809 22:48:42.201110 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.201118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.201222 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.201330 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:42.201812 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.202842 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.204772 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.205379 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.205480 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.210385 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:42.210396 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:42.210413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:42.211332 (+ 919us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:42.211409 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:42.211432 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:42.211544 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.211739 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.211853 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260809 22:48:42.212827 11173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:42.326794 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11039
I20260809 22:48:42.347985 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11184
I20260809 22:48:42.356442 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.349571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.21.27.130:36415)
0809 22:48:42.349770 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.349776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.349797 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:48:42.349867 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.349872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.350037 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.350151 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:42.350848 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.352014 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.353270 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.353692 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.353829 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.354963 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:42.354974 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:42.354992 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:42.355666 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:42.355759 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:42.355784 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:42.355893 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.355997 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.356110 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:48:42.357133 11318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:42.449409 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11184
I20260809 22:48:42.469890 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11326
I20260809 22:48:42.478190 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:42.470738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.21.27.131:45257)
0809 22:48:42.471003 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:42.471010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:42.471034 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:48:42.471102 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:42.471107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:42.471413 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:42.471551 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:42.471939 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.473161 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.474412 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:42.475135 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:42.475304 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:42.476435 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:42.476447 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:42.476468 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:42.477275 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:42.477344 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:42.477370 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:42.477488 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:42.477672 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 22:48:42.477774 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 22:48:42.478997 11460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:42.602685 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11326
I20260809 22:48:42.621089 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 10940
I20260809 22:48:42.622246 11006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:42.732810 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 10940
2026-08-09T22:48:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3103 ms)
[----------] 6 tests from EncryptionPolicyTest (22376 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:42 dist-test-slave-chsk krb5kdc[11521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:42 dist-test-slave-chsk krb5kdc[11521](info): set up 2 sockets
Aug 09 22:48:42 dist-test-slave-chsk krb5kdc[11521](info): commencing operation
krb5kdc: starting...
W20260809 22:48:43.952227 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:44 dist-test-slave-chsk krb5kdc[11521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:44.248715 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:44.580504 11536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:44.580804 11536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:44.580901 11536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:44.588179 11536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:44.588294 11536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:44.588339 11536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:44.588402 11536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:44.588445 11536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:44.598426 11536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:44.599733 11536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:44.600981 11536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:44.615672 11541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:44.615736 11545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:44.615732 11542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:44.617584 11536 server_base.cc:1034] running on GCE node
I20260809 22:48:44.618395 11536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:44.619872 11536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:44.621163 11536 hybrid_clock.cc:648] HybridClock initialized: now 1786315724621097 us; error 34 us; skew 500 ppm
I20260809 22:48:44.621517 11536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:44 dist-test-slave-chsk krb5kdc[11521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:44.625010 11536 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:44.626688 11536 webserver.cc:460] Webserver started at http://127.0.0.1:36457/ using document root <none> and password file <none>
I20260809 22:48:44.627460 11536 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:44.627610 11536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:44.627934 11536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:44.631027 11536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0aea832c7e2748dc80fe658ac2fe2ed5"
format_stamp: "Formatted at 2026-08-09 22:48:44 on dist-test-slave-chsk"
server_key: "c0ec4e87a95a4b08c15613bd63cd268f"
server_key_iv: "9f968ef3234e2a6693b7d2334b255c0b"
server_key_version: "encryptionkey@0"
I20260809 22:48:44.631901 11536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0aea832c7e2748dc80fe658ac2fe2ed5"
format_stamp: "Formatted at 2026-08-09 22:48:44 on dist-test-slave-chsk"
server_key: "c0ec4e87a95a4b08c15613bd63cd268f"
server_key_iv: "9f968ef3234e2a6693b7d2334b255c0b"
server_key_version: "encryptionkey@0"
I20260809 22:48:44.637225 11536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:48:44.640944 11552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:44.642402 11536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:48:44.642643 11536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0aea832c7e2748dc80fe658ac2fe2ed5"
format_stamp: "Formatted at 2026-08-09 22:48:44 on dist-test-slave-chsk"
server_key: "c0ec4e87a95a4b08c15613bd63cd268f"
server_key_iv: "9f968ef3234e2a6693b7d2334b255c0b"
server_key_version: "encryptionkey@0"
I20260809 22:48:44.642868 11536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:44.671883 11536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:44.676556 11536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:44.676862 11536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:44.696439 11536 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41891
I20260809 22:48:44.696565 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41891 every 8 connection(s)
I20260809 22:48:44.697968 11536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:44.701820 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:44.707129 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11536
I20260809 22:48:44.707423 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:44.708256 21614 external_mini_cluster.cc:1428] Setting key eac664ad83706122eb7c399749e70ca5
I20260809 22:48:44.713900 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5: Bootstrap starting.
I20260809 22:48:44.717753 11610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:44.719036 11610 log.cc:824] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:44.722759 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5: No bootstrap required, opened a new log
Aug 09 22:48:44 dist-test-slave-chsk krb5kdc[11521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:48:44.731079 11610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } }
I20260809 22:48:44.731514 11610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:44.731667 11610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aea832c7e2748dc80fe658ac2fe2ed5, State: Initialized, Role: FOLLOWER
I20260809 22:48:44.732396 11610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } }
I20260809 22:48:44.732677 11610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:44.732836 11610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:44.733029 11610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:44.733448 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:44.710375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53523 (local address 127.0.0.1:41891)
0809 22:48:44.711323 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:44.711335 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:44.711379 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:48:44.713113 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:44.713134 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:44.713535 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:44.713846 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:44.715815 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:44.717164 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:44.718984 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:44.720074 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:44.720281 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:44.725317 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:44.725325 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:44.725347 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:44.725388 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:44.728806 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:44.729680 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:44.729687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:44.729693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:44.729892 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:44.730387 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:44.730391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:44.730393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:44.730930 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:44.731302 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:44.731852 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 22:48:44.732223 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":427,"threads_started":1}
I20260809 22:48:44.735742 11610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } }
I20260809 22:48:44.736361 11610 leader_election.cc:302] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0aea832c7e2748dc80fe658ac2fe2ed5; no voters:
I20260809 22:48:44.737169 11610 leader_election.cc:288] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:44.737280 11616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:44.737917 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:44.739833 11610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:44.742929 11616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [term 1 LEADER]: Becoming Leader. State: Replica: 0aea832c7e2748dc80fe658ac2fe2ed5, State: Running, Role: LEADER
I20260809 22:48:44.743813 11616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } }
I20260809 22:48:44.747323 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } } }
I20260809 22:48:44.747850 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:44.748426 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0aea832c7e2748dc80fe658ac2fe2ed5. Latest consensus state: current_term: 1 leader_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0aea832c7e2748dc80fe658ac2fe2ed5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41891 } } }
I20260809 22:48:44.748793 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:44.759445 11628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:44.767081 11628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:44.770738 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:44.743234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55723 (local address 127.0.0.1:41891)
0809 22:48:44.743446 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:44.743455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:44.750250 (+ 6795us) server_negotiation.cc:407] Connection header received
0809 22:48:44.750481 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:44.750486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:44.750577 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:44.750681 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:44.753780 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:44.755394 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:44.758330 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:44.758925 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:44.759024 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:44.761852 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:44.761857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:44.761860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:44.761925 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:44.766066 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:44.767275 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:44.767282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:44.767285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:44.767428 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:44.768051 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:44.768058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:44.768060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:44.768549 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:44.769957 (+ 1408us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:44.770155 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 22:48:44.770351 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:48:44.779026 11628 catalog_manager.cc:1347] Generated new cluster ID: 9def608d8b5b475cb9cba9b71db60945
I20260809 22:48:44.779345 11628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:44.792232 11628 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:44.794119 11628 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:44.810271 11628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0aea832c7e2748dc80fe658ac2fe2ed5: Generated new TSK 0
I20260809 22:48:44.811396 11628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:44.817768 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11536
I20260809 22:48:44.819172 11604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:44.911482 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11536
W20260809 22:48:44.931671 11519 connection.cc:535] client connection to 127.0.0.1:41891 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41891): Connection reset by peer (error 104)
2026-08-09T22:48:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2167 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:45 dist-test-slave-chsk krb5kdc[11647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:45 dist-test-slave-chsk krb5kdc[11647](info): set up 2 sockets
Aug 09 22:48:45 dist-test-slave-chsk krb5kdc[11647](info): commencing operation
krb5kdc: starting...
W20260809 22:48:46.081220 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:46 dist-test-slave-chsk krb5kdc[11647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:46.386735 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:46.720364 11661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:46.720719 11661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:46.720883 11661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:46.727954 11661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:46.728124 11661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:46.728272 11661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:46.728394 11661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:46.728461 11661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:46.737990 11661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:46.739260 11661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:46.740562 11661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:46.752696 11668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:46.752734 11667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:46.752789 11672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:46.753870 11661 server_base.cc:1034] running on GCE node
I20260809 22:48:46.754659 11661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:46.756217 11661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:46.757426 11661 hybrid_clock.cc:648] HybridClock initialized: now 1786315726757378 us; error 35 us; skew 500 ppm
I20260809 22:48:46.757778 11661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:46 dist-test-slave-chsk krb5kdc[11647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:46.760974 11661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:46.762517 11661 webserver.cc:460] Webserver started at http://127.0.0.1:43543/ using document root <none> and password file <none>
I20260809 22:48:46.763240 11661 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:46.763373 11661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:46.763684 11661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:46.766700 11661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "acf7b72936144027a0a742c0c3f0f76e"
format_stamp: "Formatted at 2026-08-09 22:48:46 on dist-test-slave-chsk"
server_key: "495181d44a2deb3bde3f7a4e8430df2e"
server_key_iv: "f467eaff1a23461c31bd38632cbbba91"
server_key_version: "encryptionkey@0"
I20260809 22:48:46.767542 11661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "acf7b72936144027a0a742c0c3f0f76e"
format_stamp: "Formatted at 2026-08-09 22:48:46 on dist-test-slave-chsk"
server_key: "495181d44a2deb3bde3f7a4e8430df2e"
server_key_iv: "f467eaff1a23461c31bd38632cbbba91"
server_key_version: "encryptionkey@0"
I20260809 22:48:46.772502 11661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:48:46.775807 11679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:46.776942 11661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 22:48:46.777122 11661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "acf7b72936144027a0a742c0c3f0f76e"
format_stamp: "Formatted at 2026-08-09 22:48:46 on dist-test-slave-chsk"
server_key: "495181d44a2deb3bde3f7a4e8430df2e"
server_key_iv: "f467eaff1a23461c31bd38632cbbba91"
server_key_version: "encryptionkey@0"
I20260809 22:48:46.777309 11661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:46.800284 11661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:46.804380 11661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:46.804687 11661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:46.823858 11661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36459
I20260809 22:48:46.823902 11736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36459 every 8 connection(s)
I20260809 22:48:46.825227 11661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:46.826812 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11661
I20260809 22:48:46.827020 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:46.827625 21614 external_mini_cluster.cc:1428] Setting key 637babfe6007c111f4155064ae1af504
I20260809 22:48:46.829759 11737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:46.841463 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:46.829407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60311 (local address 127.0.0.1:36459)
0809 22:48:46.830217 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:46.830227 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:46.830266 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:48:46.831991 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:46.832012 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:46.832380 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:46.832699 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:46.833302 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:46.834667 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:46.836565 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:46.837195 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:46.837342 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:46.838772 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:46.838781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:46.838797 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:46.839419 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:46.839501 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:46.839532 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:46.839720 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:46.840270 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 22:48:46.840460 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":392,"threads_started":1}
I20260809 22:48:46.842746 11737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e: Bootstrap starting.
I20260809 22:48:46.845535 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:46.846191 11737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:46.847266 11737 log.cc:824] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:46.850746 11737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e: No bootstrap required, opened a new log
I20260809 22:48:46.857210 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:46.850361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39217 (local address 127.0.0.1:36459)
0809 22:48:46.850591 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:46.850599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:46.850992 (+ 393us) server_negotiation.cc:407] Connection header received
0809 22:48:46.851239 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:46.851243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:46.851307 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:46.851421 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:46.851986 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:46.852815 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:46.854160 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:46.854603 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:46.854771 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:46.856091 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:46.856096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:46.856116 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:46.856145 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:46.856229 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:46.856251 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:46.856397 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:46.856724 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 22:48:46.856804 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 22:48:46.859509 11737 raft_consensus.cc:348] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260809 22:48:46.859884 11737 raft_consensus.cc:374] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:46.860004 11737 raft_consensus.cc:729] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf7b72936144027a0a742c0c3f0f76e, State: Initialized, Role: FOLLOWER
I20260809 22:48:46.860597 11737 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260809 22:48:46.860795 11737 raft_consensus.cc:388] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:46.860948 11737 raft_consensus.cc:482] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:46.861123 11737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:46.863538 11737 raft_consensus.cc:504] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260809 22:48:46.864066 11737 leader_election.cc:302] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf7b72936144027a0a742c0c3f0f76e; no voters:
I20260809 22:48:46.864836 11737 leader_election.cc:288] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:46.864982 11747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:46.865667 11747 raft_consensus.cc:686] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [term 1 LEADER]: Becoming Leader. State: Replica: acf7b72936144027a0a742c0c3f0f76e, State: Running, Role: LEADER
I20260809 22:48:46.866281 11747 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } }
I20260809 22:48:46.867172 11737 sys_catalog.cc:564] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:46.869318 11748 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf7b72936144027a0a742c0c3f0f76e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } } }
I20260809 22:48:46.870443 11748 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:46.869930 11749 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf7b72936144027a0a742c0c3f0f76e. Latest consensus state: current_term: 1 leader_uuid: "acf7b72936144027a0a742c0c3f0f76e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf7b72936144027a0a742c0c3f0f76e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36459 } } }
I20260809 22:48:46.870769 11749 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:46.872224 11755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:46.877846 11755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:46.887072 11755 catalog_manager.cc:1347] Generated new cluster ID: 0534c113c71641d3a74ce10ad7d5ba77
I20260809 22:48:46.887287 11755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:46.913074 11755 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:46.914618 11755 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:46.924538 11755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acf7b72936144027a0a742c0c3f0f76e: Generated new TSK 0
I20260809 22:48:46.925433 11755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:46.933280 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11661
W20260809 22:48:46.934510 11730 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60311
W20260809 22:48:46.935367 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:48:46.935560 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11661
W20260809 22:48:46.957902 11644 connection.cc:535] client connection to 127.0.0.1:36459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36459): Connection reset by peer (error 104)
2026-08-09T22:48:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:47 dist-test-slave-chsk krb5kdc[11770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:47 dist-test-slave-chsk krb5kdc[11770](info): set up 2 sockets
Aug 09 22:48:47 dist-test-slave-chsk krb5kdc[11770](info): commencing operation
krb5kdc: starting...
W20260809 22:48:48.119355 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:48 dist-test-slave-chsk krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:48.422190 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:48.756903 11785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:48.757176 11785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:48.757344 11785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:48.764484 11785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:48.764611 11785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:48.764716 11785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:48.764819 11785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:48.764904 11785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:48.774480 11785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45125
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:48.775656 11785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:48.776889 11785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:48.788584 11794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:48.788674 11791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:48.788647 11790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:48.789568 11785 server_base.cc:1034] running on GCE node
I20260809 22:48:48.790452 11785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:48.791841 11785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:48.793069 11785 hybrid_clock.cc:648] HybridClock initialized: now 1786315728793032 us; error 34 us; skew 500 ppm
I20260809 22:48:48.793416 11785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:48 dist-test-slave-chsk krb5kdc[11770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315728, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:48.801390 11785 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:48.802958 11785 webserver.cc:460] Webserver started at http://127.0.0.1:33259/ using document root <none> and password file <none>
I20260809 22:48:48.803702 11785 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:48.803857 11785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:48.804193 11785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:48.807287 11785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0d2c49fa31c24d00a5fc6640f739bae0"
format_stamp: "Formatted at 2026-08-09 22:48:48 on dist-test-slave-chsk"
server_key: "1c637f7256ea52f5da11566b73e9ce76"
server_key_iv: "f2643d5a210a50d8763eebec980438f0"
server_key_version: "encryptionkey@0"
I20260809 22:48:48.808066 11785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0d2c49fa31c24d00a5fc6640f739bae0"
format_stamp: "Formatted at 2026-08-09 22:48:48 on dist-test-slave-chsk"
server_key: "1c637f7256ea52f5da11566b73e9ce76"
server_key_iv: "f2643d5a210a50d8763eebec980438f0"
server_key_version: "encryptionkey@0"
I20260809 22:48:48.813288 11785 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:48:48.816962 11803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:48.818219 11785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:48:48.818408 11785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0d2c49fa31c24d00a5fc6640f739bae0"
format_stamp: "Formatted at 2026-08-09 22:48:48 on dist-test-slave-chsk"
server_key: "1c637f7256ea52f5da11566b73e9ce76"
server_key_iv: "f2643d5a210a50d8763eebec980438f0"
server_key_version: "encryptionkey@0"
I20260809 22:48:48.818610 11785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:48.824589 11785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:48.828533 11785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:48.828835 11785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:48.848026 11785 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45125
I20260809 22:48:48.848080 11859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45125 every 8 connection(s)
I20260809 22:48:48.849468 11785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:48.851471 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11785
I20260809 22:48:48.851673 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:48.852253 21614 external_mini_cluster.cc:1428] Setting key 364955587cc078dff03b7c4159c3e45c
I20260809 22:48:48.853951 11860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:48.860042 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:48.854064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36315 (local address 127.0.0.1:45125)
0809 22:48:48.854927 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:48.854939 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:48.854980 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:48:48.856633 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:48.856653 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:48.856986 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:48.857314 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:48.858409 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:48.858415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:48.858425 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:48.858441 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:48.858509 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:48.858527 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:48.858688 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:48.858836 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 22:48:48.859043 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":438,"threads_started":1}
I20260809 22:48:48.863802 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:48.866567 11860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0: Bootstrap starting.
I20260809 22:48:48.870388 11860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:48.870735 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:48.868239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60311 (local address 127.0.0.1:45125)
0809 22:48:48.868450 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:48.868457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:48.868809 (+ 352us) server_negotiation.cc:407] Connection header received
0809 22:48:48.868970 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:48.868974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:48.869020 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:48.869109 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:48.870061 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:48.870064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:48.870068 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:48.870082 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:48.870128 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:48.870142 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:48.870227 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:48.870400 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 22:48:48.870447 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 22:48:48.871730 11860 log.cc:824] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:48.874465 11860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0: No bootstrap required, opened a new log
I20260809 22:48:48.882266 11860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260809 22:48:48.882607 11860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:48.882681 11860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2c49fa31c24d00a5fc6640f739bae0, State: Initialized, Role: FOLLOWER
I20260809 22:48:48.883285 11860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260809 22:48:48.883489 11860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:48.883591 11860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:48.883718 11860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:48.885881 11860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260809 22:48:48.886329 11860 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2c49fa31c24d00a5fc6640f739bae0; no voters:
I20260809 22:48:48.886884 11860 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:48.887005 11871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:48.887724 11871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [term 1 LEADER]: Becoming Leader. State: Replica: 0d2c49fa31c24d00a5fc6640f739bae0, State: Running, Role: LEADER
I20260809 22:48:48.888418 11871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } }
I20260809 22:48:48.889457 11860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:48.891508 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2c49fa31c24d00a5fc6640f739bae0. Latest consensus state: current_term: 1 leader_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260809 22:48:48.891987 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:48.892552 11872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2c49fa31c24d00a5fc6640f739bae0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45125 } } }
I20260809 22:48:48.892917 11872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:48.900130 11878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:48.905454 11878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:48.914254 11878 catalog_manager.cc:1347] Generated new cluster ID: 48b056652da64888a6d5d4f03c7c8d85
I20260809 22:48:48.914390 11878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:48.942919 11878 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:48.943436 11878 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:48.950513 11878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2c49fa31c24d00a5fc6640f739bae0: Generated new TSK 0
I20260809 22:48:48.951380 11878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:48.956197 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11785
W20260809 22:48:48.957264 11855 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36315
W20260809 22:48:48.958127 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11785: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:48:48.958283 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11785
2026-08-09T22:48:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2019 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:49 dist-test-slave-chsk krb5kdc[11892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:49 dist-test-slave-chsk krb5kdc[11892](info): set up 2 sockets
Aug 09 22:48:49 dist-test-slave-chsk krb5kdc[11892](info): commencing operation
krb5kdc: starting...
W20260809 22:48:50.133328 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:50 dist-test-slave-chsk krb5kdc[11892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:50.459208 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:50.815812 11906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:50.816152 11906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:50.816380 11906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:50.824046 11906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:50.824185 11906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:50.824311 11906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:50.824417 11906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:50.824503 11906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:50.834875 11906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:50.836385 11906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:50.837749 11906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:50.850152 11914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:50.850319 11911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:50.850296 11912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:50.851491 11906 server_base.cc:1034] running on GCE node
I20260809 22:48:50.852381 11906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:50.853889 11906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:50.855182 11906 hybrid_clock.cc:648] HybridClock initialized: now 1786315730855108 us; error 53 us; skew 500 ppm
I20260809 22:48:50.855587 11906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:50 dist-test-slave-chsk krb5kdc[11892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315730, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:50.859043 11906 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:50.860879 11906 webserver.cc:460] Webserver started at http://127.0.0.1:46743/ using document root <none> and password file <none>
I20260809 22:48:50.861711 11906 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:50.861883 11906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:50.862241 11906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:50.865480 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "497a82803a1a4120b42867863577634e"
format_stamp: "Formatted at 2026-08-09 22:48:50 on dist-test-slave-chsk"
server_key: "eec1b581ce7d239cb209ba599a505df5"
server_key_iv: "47d1b24692bc669e0385288cdc03e705"
server_key_version: "encryptionkey@0"
I20260809 22:48:50.866340 11906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "497a82803a1a4120b42867863577634e"
format_stamp: "Formatted at 2026-08-09 22:48:50 on dist-test-slave-chsk"
server_key: "eec1b581ce7d239cb209ba599a505df5"
server_key_iv: "47d1b24692bc669e0385288cdc03e705"
server_key_version: "encryptionkey@0"
I20260809 22:48:50.872538 11906 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:48:50.876377 11924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:50.877823 11906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:48:50.878036 11906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "497a82803a1a4120b42867863577634e"
format_stamp: "Formatted at 2026-08-09 22:48:50 on dist-test-slave-chsk"
server_key: "eec1b581ce7d239cb209ba599a505df5"
server_key_iv: "47d1b24692bc669e0385288cdc03e705"
server_key_version: "encryptionkey@0"
I20260809 22:48:50.878258 11906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:50.895008 11906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:50.899274 11906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:50.899588 11906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:50.919646 11906 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42571
I20260809 22:48:50.919677 11981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42571 every 8 connection(s)
I20260809 22:48:50.921168 11906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:50.922356 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 11906
I20260809 22:48:50.922601 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:50.923059 21614 external_mini_cluster.cc:1428] Setting key c4eb9fabe45709b698239073b07a77df
I20260809 22:48:50.925941 11982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:48:50 dist-test-slave-chsk krb5kdc[11892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:48:50.939564 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: Bootstrap starting.
I20260809 22:48:50.943236 11982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:50.944319 11982 log.cc:824] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:50.947474 11982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: No bootstrap required, opened a new log
I20260809 22:48:50.947515 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:50.924976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42291 (local address 127.0.0.1:42571)
0809 22:48:50.925839 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:50.925859 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:50.925929 (+ 70us) server_negotiation.cc:407] Connection header received
0809 22:48:50.928394 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:50.928424 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:50.928929 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:50.929399 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:50.930856 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:50.932990 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:50.934256 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:50.935324 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:50.935542 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:50.939943 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:50.939951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:50.939966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:50.940009 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:50.943120 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:50.943948 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:50.943963 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:50.943970 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:50.944160 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:50.944589 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:50.944594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:50.944599 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:50.945180 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:50.945559 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:50.946015 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 22:48:50.946457 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":354,"threads_started":1}
I20260809 22:48:50.951651 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:50.956221 11982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260809 22:48:50.956765 11982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:50.956877 11982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497a82803a1a4120b42867863577634e, State: Initialized, Role: FOLLOWER
I20260809 22:48:50.957589 11982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260809 22:48:50.957911 11982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:50.958077 11982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:50.958284 11982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:50.961520 11982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260809 22:48:50.962280 11982 leader_election.cc:302] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 497a82803a1a4120b42867863577634e; no voters:
I20260809 22:48:50.963017 11982 leader_election.cc:288] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:50.963186 11993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:50.966091 11982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:50.967350 11993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [term 1 LEADER]: Becoming Leader. State: Replica: 497a82803a1a4120b42867863577634e, State: Running, Role: LEADER
I20260809 22:48:50.968657 11993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260809 22:48:50.977030 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:50.956271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43913 (local address 127.0.0.1:42571)
0809 22:48:50.956519 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:50.956529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:50.956968 (+ 439us) server_negotiation.cc:407] Connection header received
0809 22:48:50.957204 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:50.957209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:50.957325 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:50.957429 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:50.959708 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:50.960512 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:50.962271 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:50.962881 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:50.963043 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:50.967346 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:50.967355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:50.967360 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:50.967417 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:50.972468 (+ 5051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:50.973856 (+ 1388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:50.973867 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:50.973873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:50.974168 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:50.974844 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:50.974852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:50.974858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:50.975448 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:50.975843 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:50.976187 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:48:50.976549 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:48:50.978158 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "497a82803a1a4120b42867863577634e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } } }
I20260809 22:48:50.978657 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:50.979241 11995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 497a82803a1a4120b42867863577634e. Latest consensus state: current_term: 1 leader_uuid: "497a82803a1a4120b42867863577634e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497a82803a1a4120b42867863577634e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } } }
I20260809 22:48:50.979553 11995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e [sys.catalog]: This master's current role is: LEADER
W20260809 22:48:50.984637 12012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:48:50.984797 12012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:48:50.984962 12010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:50.986680 12010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:50.995287 12010 catalog_manager.cc:1347] Generated new cluster ID: 8c0dd52a98064a9cb5fe5eac62ce1095
I20260809 22:48:50.995424 12010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:51.013404 12010 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:51.015086 12010 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:51.025086 12010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 497a82803a1a4120b42867863577634e: Generated new TSK 0
I20260809 22:48:51.026150 12010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:51.046008 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 11906
I20260809 22:48:51.047513 11977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:51.161906 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 11906
2026-08-09T22:48:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:51 dist-test-slave-chsk krb5kdc[12018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:51 dist-test-slave-chsk krb5kdc[12018](info): set up 2 sockets
Aug 09 22:48:51 dist-test-slave-chsk krb5kdc[12018](info): commencing operation
krb5kdc: starting...
W20260809 22:48:52.349475 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:52 dist-test-slave-chsk krb5kdc[12018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:52.666577 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:53.007882 12032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:53.008216 12032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:53.008313 12032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:53.016482 12032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:53.016615 12032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:53.016664 12032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:53.016721 12032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:53.016758 12032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:53.027746 12032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:53.029091 12032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:53.030424 12032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:53.042454 12038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:53.042497 12041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:53.042531 12039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:53.043788 12032 server_base.cc:1034] running on GCE node
I20260809 22:48:53.044590 12032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:53.046032 12032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:53.047259 12032 hybrid_clock.cc:648] HybridClock initialized: now 1786315733047203 us; error 47 us; skew 500 ppm
I20260809 22:48:53.047616 12032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:53 dist-test-slave-chsk krb5kdc[12018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:53.050844 12032 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:53.052404 12032 webserver.cc:460] Webserver started at http://127.0.0.1:46511/ using document root <none> and password file <none>
I20260809 22:48:53.053045 12032 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:53.053149 12032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:53.053398 12032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:53.056582 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "a5067bfd1495406e9d4ee188dac6a547"
format_stamp: "Formatted at 2026-08-09 22:48:53 on dist-test-slave-chsk"
server_key: "7d314f28bfc64969d9a5993b37b69862"
server_key_iv: "c1a5e3349534b79946a310fd7263b9ed"
server_key_version: "encryptionkey@0"
I20260809 22:48:53.057462 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "a5067bfd1495406e9d4ee188dac6a547"
format_stamp: "Formatted at 2026-08-09 22:48:53 on dist-test-slave-chsk"
server_key: "7d314f28bfc64969d9a5993b37b69862"
server_key_iv: "c1a5e3349534b79946a310fd7263b9ed"
server_key_version: "encryptionkey@0"
I20260809 22:48:53.062572 12032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 22:48:53.066008 12048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:53.067356 12032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:48:53.067494 12032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "a5067bfd1495406e9d4ee188dac6a547"
format_stamp: "Formatted at 2026-08-09 22:48:53 on dist-test-slave-chsk"
server_key: "7d314f28bfc64969d9a5993b37b69862"
server_key_iv: "c1a5e3349534b79946a310fd7263b9ed"
server_key_version: "encryptionkey@0"
I20260809 22:48:53.067651 12032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:53.079617 12032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:53.083765 12032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:53.084003 12032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:53.103339 12032 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42937
I20260809 22:48:53.103389 12111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42937 every 8 connection(s)
I20260809 22:48:53.104764 12032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:53.105023 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12032
I20260809 22:48:53.105238 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:53.105924 21614 external_mini_cluster.cc:1428] Setting key 571b650295ec6343f38fb3111d9cb248
I20260809 22:48:53.109632 12112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:53.121299 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:53.107767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38573 (local address 127.0.0.1:42937)
0809 22:48:53.108555 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:53.108570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:53.108618 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:48:53.110956 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:53.110983 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:53.111495 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:53.111923 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:53.112435 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:53.114426 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:53.115882 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:53.116911 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:53.117082 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:53.118354 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:53.118366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:53.118386 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:53.119108 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:53.119250 (+ 142us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:53.119282 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:53.119396 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:53.119623 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 22:48:53.119911 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":330,"threads_started":1}
I20260809 22:48:53.123454 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547: Bootstrap starting.
I20260809 22:48:53.125072 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:53.127228 12112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:53.128597 12112 log.cc:824] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:53.131978 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547: No bootstrap required, opened a new log
I20260809 22:48:53.137231 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:53.130435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39649 (local address 127.0.0.1:42937)
0809 22:48:53.130617 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:53.130623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:53.131176 (+ 553us) server_negotiation.cc:407] Connection header received
0809 22:48:53.131367 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:53.131374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:53.131462 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:53.131583 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:53.132114 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:53.133216 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:53.134511 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:53.135298 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:53.135430 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:53.136438 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:53.136445 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:53.136449 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:53.136469 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:53.136516 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:53.136535 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:53.136640 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:53.136756 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 22:48:53.136860 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:48:53.140868 12112 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } }
I20260809 22:48:53.141263 12112 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:53.141386 12112 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5067bfd1495406e9d4ee188dac6a547, State: Initialized, Role: FOLLOWER
I20260809 22:48:53.142027 12112 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } }
I20260809 22:48:53.142264 12112 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:53.142422 12112 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:53.142603 12112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:53.144961 12112 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } }
I20260809 22:48:53.145467 12112 leader_election.cc:302] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5067bfd1495406e9d4ee188dac6a547; no voters:
I20260809 22:48:53.146102 12112 leader_election.cc:288] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:53.146232 12123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:53.147554 12123 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [term 1 LEADER]: Becoming Leader. State: Replica: a5067bfd1495406e9d4ee188dac6a547, State: Running, Role: LEADER
I20260809 22:48:53.148123 12123 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } }
I20260809 22:48:53.148859 12112 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:53.151163 12124 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5067bfd1495406e9d4ee188dac6a547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } } }
I20260809 22:48:53.151613 12124 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:53.152325 12125 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5067bfd1495406e9d4ee188dac6a547. Latest consensus state: current_term: 1 leader_uuid: "a5067bfd1495406e9d4ee188dac6a547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5067bfd1495406e9d4ee188dac6a547" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42937 } } }
I20260809 22:48:53.152595 12131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:53.152595 12125 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:53.160049 12131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:53.169247 12131 catalog_manager.cc:1347] Generated new cluster ID: 1ad2cf6d20604761b00d1b861b6f3b56
I20260809 22:48:53.169382 12131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:53.201524 12131 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:53.203567 12131 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:53.219331 12131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5067bfd1495406e9d4ee188dac6a547: Generated new TSK 0
I20260809 22:48:53.220386 12131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:53.233675 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12032
W20260809 22:48:53.235260 12107 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38573
W20260809 22:48:53.236006 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12032: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:48:53.236159 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12032
2026-08-09T22:48:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2073 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:53 dist-test-slave-chsk krb5kdc[12145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:53 dist-test-slave-chsk krb5kdc[12145](info): set up 2 sockets
Aug 09 22:48:53 dist-test-slave-chsk krb5kdc[12145](info): commencing operation
krb5kdc: starting...
W20260809 22:48:54.412081 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:54 dist-test-slave-chsk krb5kdc[12145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:54.714123 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:55.056892 12160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:55.057237 12160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:55.057400 12160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:55.064821 12160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:55.064970 12160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:55.065080 12160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:55.065192 12160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:55.065271 12160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:55.075126 12160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:55.076529 12160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:55.077816 12160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:55.089604 12165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:55.089555 12168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:55.089541 12166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:55.090190 12160 server_base.cc:1034] running on GCE node
I20260809 22:48:55.091032 12160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:55.092444 12160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:55.093681 12160 hybrid_clock.cc:648] HybridClock initialized: now 1786315735093655 us; error 51 us; skew 500 ppm
I20260809 22:48:55.094026 12160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:55 dist-test-slave-chsk krb5kdc[12145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:55.103091 12160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:55.104671 12160 webserver.cc:460] Webserver started at http://127.0.0.1:34567/ using document root <none> and password file <none>
I20260809 22:48:55.105350 12160 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:55.105504 12160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:55.105819 12160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:55.108816 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "ad1861a3c183440792da4be04931407c"
format_stamp: "Formatted at 2026-08-09 22:48:55 on dist-test-slave-chsk"
server_key: "15542c0c2dc64f864469e0e550f47f8d"
server_key_iv: "b3a7750620a9c31dc03cdd9455c7675d"
server_key_version: "encryptionkey@0"
I20260809 22:48:55.109589 12160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "ad1861a3c183440792da4be04931407c"
format_stamp: "Formatted at 2026-08-09 22:48:55 on dist-test-slave-chsk"
server_key: "15542c0c2dc64f864469e0e550f47f8d"
server_key_iv: "b3a7750620a9c31dc03cdd9455c7675d"
server_key_version: "encryptionkey@0"
I20260809 22:48:55.114687 12160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:48:55.118130 12175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:55.119410 12160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:48:55.119592 12160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "ad1861a3c183440792da4be04931407c"
format_stamp: "Formatted at 2026-08-09 22:48:55 on dist-test-slave-chsk"
server_key: "15542c0c2dc64f864469e0e550f47f8d"
server_key_iv: "b3a7750620a9c31dc03cdd9455c7675d"
server_key_version: "encryptionkey@0"
I20260809 22:48:55.119797 12160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:55.125686 12160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:55.129663 12160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:55.129982 12160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:55.148626 12160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46179
I20260809 22:48:55.148679 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46179 every 8 connection(s)
I20260809 22:48:55.149961 12160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:55.153525 12233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:55.155750 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12160
I20260809 22:48:55.155951 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:55.156384 21614 external_mini_cluster.cc:1428] Setting key 3f7e062607ec65ac6e43cacf7ade55a7
I20260809 22:48:55.164016 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:55.158083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40293 (local address 127.0.0.1:46179)
0809 22:48:55.158890 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:55.158901 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:55.158948 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:48:55.160603 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:55.160624 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:55.160930 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:55.161316 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:55.162434 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:55.162440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:55.162455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:55.162472 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:55.162528 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:55.162546 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:55.162725 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:55.162830 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 22:48:55.163022 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":405,"threads_started":1}
I20260809 22:48:55.165632 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c: Bootstrap starting.
I20260809 22:48:55.167717 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:55.169001 12233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:55.170004 12233 log.cc:824] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:55.173341 12233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c: No bootstrap required, opened a new log
I20260809 22:48:55.174835 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:55.172240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52981 (local address 127.0.0.1:46179)
0809 22:48:55.172417 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:55.172425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:55.172848 (+ 423us) server_negotiation.cc:407] Connection header received
0809 22:48:55.173041 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:55.173047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:55.173101 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:55.173202 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:55.174182 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:55.174188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:55.174192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:55.174217 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:55.174263 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:55.174278 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:55.174405 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:55.174501 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 22:48:55.174561 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:48:55.181437 12233 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } }
I20260809 22:48:55.181833 12233 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:55.181942 12233 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1861a3c183440792da4be04931407c, State: Initialized, Role: FOLLOWER
I20260809 22:48:55.182505 12233 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } }
I20260809 22:48:55.182762 12233 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:55.182915 12233 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:55.183089 12233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:55.185987 12233 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } }
I20260809 22:48:55.186486 12233 leader_election.cc:302] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1861a3c183440792da4be04931407c; no voters:
I20260809 22:48:55.187207 12233 leader_election.cc:288] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:55.187332 12244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:55.188020 12244 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [term 1 LEADER]: Becoming Leader. State: Replica: ad1861a3c183440792da4be04931407c, State: Running, Role: LEADER
I20260809 22:48:55.188673 12244 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } }
I20260809 22:48:55.189728 12233 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:55.191443 12246 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1861a3c183440792da4be04931407c. Latest consensus state: current_term: 1 leader_uuid: "ad1861a3c183440792da4be04931407c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } } }
I20260809 22:48:55.191862 12246 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:55.192523 12247 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1861a3c183440792da4be04931407c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1861a3c183440792da4be04931407c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46179 } } }
I20260809 22:48:55.192962 12247 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:55.200408 12250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:55.205753 12250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:55.214970 12250 catalog_manager.cc:1347] Generated new cluster ID: 3a49045a50a549d4939bf00c5027843a
I20260809 22:48:55.215106 12250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:55.253369 12250 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:55.253820 12250 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:55.261108 12250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1861a3c183440792da4be04931407c: Generated new TSK 0
I20260809 22:48:55.261962 12250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:55.274325 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12160
W20260809 22:48:55.275619 12228 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40293
W20260809 22:48:55.276369 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12160: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:48:55.276530 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12160
2026-08-09T22:48:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:55 dist-test-slave-chsk krb5kdc[12267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:55 dist-test-slave-chsk krb5kdc[12267](info): set up 2 sockets
Aug 09 22:48:55 dist-test-slave-chsk krb5kdc[12267](info): commencing operation
krb5kdc: starting...
W20260809 22:48:56.454689 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:56 dist-test-slave-chsk krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:56.758088 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:57.093379 12281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:57.093640 12281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:57.093734 12281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:57.100867 12281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:57.100984 12281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:57.101038 12281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:57.101092 12281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:57.101138 12281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:57.110491 12281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:57.111601 12281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:57.112847 12281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:57.124168 12286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:57.124205 12289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:57.124305 12287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:57.125278 12281 server_base.cc:1034] running on GCE node
I20260809 22:48:57.125982 12281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:57.127395 12281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:57.128604 12281 hybrid_clock.cc:648] HybridClock initialized: now 1786315737128560 us; error 31 us; skew 500 ppm
I20260809 22:48:57.128942 12281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:57 dist-test-slave-chsk krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315737, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:57.131942 12281 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:57.133352 12281 webserver.cc:460] Webserver started at http://127.0.0.1:45177/ using document root <none> and password file <none>
I20260809 22:48:57.133934 12281 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:57.134037 12281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:57.134282 12281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:57.137117 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "d5073ac58b39417ebcb51d42df48a77b"
format_stamp: "Formatted at 2026-08-09 22:48:57 on dist-test-slave-chsk"
server_key: "a79dd956651c731ea21db2fb9ba515fe"
server_key_iv: "e2acd5592d4b5aa83d91c0a17f708340"
server_key_version: "encryptionkey@0"
I20260809 22:48:57.137805 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "d5073ac58b39417ebcb51d42df48a77b"
format_stamp: "Formatted at 2026-08-09 22:48:57 on dist-test-slave-chsk"
server_key: "a79dd956651c731ea21db2fb9ba515fe"
server_key_iv: "e2acd5592d4b5aa83d91c0a17f708340"
server_key_version: "encryptionkey@0"
I20260809 22:48:57.142699 12281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 22:48:57.145881 12297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:57.147068 12281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 22:48:57.147269 12281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "d5073ac58b39417ebcb51d42df48a77b"
format_stamp: "Formatted at 2026-08-09 22:48:57 on dist-test-slave-chsk"
server_key: "a79dd956651c731ea21db2fb9ba515fe"
server_key_iv: "e2acd5592d4b5aa83d91c0a17f708340"
server_key_version: "encryptionkey@0"
I20260809 22:48:57.147454 12281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:57.168991 12281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:57.172549 12281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:57.172820 12281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:57.191049 12281 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44653
I20260809 22:48:57.191102 12356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44653 every 8 connection(s)
I20260809 22:48:57.192343 12281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:57.195760 12357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:57.199226 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12281
I20260809 22:48:57.199424 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:57.199879 21614 external_mini_cluster.cc:1428] Setting key 8db7f37c4f365934883798d1b18f3fd4
I20260809 22:48:57.207913 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b: Bootstrap starting.
I20260809 22:48:57.211555 12357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:57.212842 12357 log.cc:824] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:48:57 dist-test-slave-chsk krb5kdc[12267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:48:57.215718 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b: No bootstrap required, opened a new log
I20260809 22:48:57.223999 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:57.201529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:52777 (local address 127.0.0.1:44653)
0809 22:48:57.202241 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:57.202251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:57.202288 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:48:57.203955 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:57.203974 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:57.204352 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:57.204638 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:57.206373 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:57.207700 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:57.209569 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:57.210217 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:57.210339 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:57.215411 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:57.215418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:57.215430 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:57.215462 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:57.219586 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:57.220317 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:57.220326 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:57.220333 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:57.220538 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:57.220973 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:57.220984 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:57.220988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:57.221651 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:57.221962 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:57.222377 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 22:48:57.222693 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":315,"threads_started":1}
I20260809 22:48:57.226130 12357 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260809 22:48:57.226540 12357 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:57.226637 12357 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5073ac58b39417ebcb51d42df48a77b, State: Initialized, Role: FOLLOWER
I20260809 22:48:57.227332 12357 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260809 22:48:57.227586 12357 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:57.227748 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:57.227751 12357 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:57.227916 12357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:57.230958 12357 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260809 22:48:57.231601 12357 leader_election.cc:302] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5073ac58b39417ebcb51d42df48a77b; no voters:
I20260809 22:48:57.232306 12357 leader_election.cc:288] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:57.232409 12367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:57.234704 12367 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [term 1 LEADER]: Becoming Leader. State: Replica: d5073ac58b39417ebcb51d42df48a77b, State: Running, Role: LEADER
I20260809 22:48:57.234766 12357 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:57.235368 12367 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } }
I20260809 22:48:57.241633 12368 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5073ac58b39417ebcb51d42df48a77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } } }
I20260809 22:48:57.242115 12368 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:57.242761 12369 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5073ac58b39417ebcb51d42df48a77b. Latest consensus state: current_term: 1 leader_uuid: "d5073ac58b39417ebcb51d42df48a77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5073ac58b39417ebcb51d42df48a77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44653 } } }
I20260809 22:48:57.243069 12369 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:57.258122 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:57.233768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:54477 (local address 127.0.0.1:44653)
0809 22:48:57.234049 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:57.234058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:57.236928 (+ 2870us) server_negotiation.cc:407] Connection header received
0809 22:48:57.237126 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:57.237135 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:57.237236 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:57.237368 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:57.238923 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:57.240189 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:57.242071 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:57.242885 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:57.243078 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:57.247390 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:57.247400 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:57.247407 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:48:57.247456 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:57.252675 (+ 5219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:57.253917 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:57.253926 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:57.253931 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:57.254119 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:48:57.254819 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:48:57.254827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:48:57.254832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:48:57.255402 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:57.255715 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:57.257351 (+ 1636us) server_negotiation.cc:299] Negotiation successful
0809 22:48:57.257599 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 22:48:57.260159 12380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:57.264704 12380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:57.272811 12380 catalog_manager.cc:1347] Generated new cluster ID: c639972869664f1598463811c900ab2b
I20260809 22:48:57.272917 12380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:57.292980 12380 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:57.294777 12380 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:57.307798 12380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5073ac58b39417ebcb51d42df48a77b: Generated new TSK 0
I20260809 22:48:57.308836 12380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:57.315207 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12281
I20260809 22:48:57.316457 12351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:48:57.410705 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12281
W20260809 22:48:57.431102 12265 connection.cc:535] client connection to 127.0.0.1:44653 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44653): Connection reset by peer (error 104)
2026-08-09T22:48:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2133 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:57 dist-test-slave-chsk krb5kdc[12394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:57 dist-test-slave-chsk krb5kdc[12394](info): set up 2 sockets
Aug 09 22:48:57 dist-test-slave-chsk krb5kdc[12394](info): commencing operation
krb5kdc: starting...
W20260809 22:48:58.583184 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:48:58 dist-test-slave-chsk krb5kdc[12394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:48:58.919078 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:48:59.266788 12409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:48:59.267200 12409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:48:59.267400 12409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:48:59.274906 12409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:48:59.275084 12409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:48:59.275218 12409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:48:59.275331 12409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:48:59.275408 12409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:48:59.286170 12409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:48:59.287633 12409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:48:59.288975 12409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:48:59.301445 12415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:59.301481 12417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:48:59.301545 12414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:48:59.302372 12409 server_base.cc:1034] running on GCE node
I20260809 22:48:59.303238 12409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:48:59.304744 12409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:48:59.305994 12409 hybrid_clock.cc:648] HybridClock initialized: now 1786315739305955 us; error 64 us; skew 500 ppm
I20260809 22:48:59.306396 12409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:48:59 dist-test-slave-chsk krb5kdc[12394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315739, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:48:59.309775 12409 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:48:59.311347 12409 webserver.cc:460] Webserver started at http://127.0.0.1:43029/ using document root <none> and password file <none>
I20260809 22:48:59.311988 12409 fs_manager.cc:359] Metadata directory not provided
I20260809 22:48:59.312101 12409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:48:59.312353 12409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:48:59.315441 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "0522e2b3ecda4c0fba11b580b28c1378"
format_stamp: "Formatted at 2026-08-09 22:48:59 on dist-test-slave-chsk"
server_key: "84198a59f35d14bd4218fa4df4678751"
server_key_iv: "8ef57b7cb701725991c705d4d9ea616e"
server_key_version: "encryptionkey@0"
I20260809 22:48:59.316211 12409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "0522e2b3ecda4c0fba11b580b28c1378"
format_stamp: "Formatted at 2026-08-09 22:48:59 on dist-test-slave-chsk"
server_key: "84198a59f35d14bd4218fa4df4678751"
server_key_iv: "8ef57b7cb701725991c705d4d9ea616e"
server_key_version: "encryptionkey@0"
I20260809 22:48:59.321787 12409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:48:59.325459 12424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:48:59.326838 12409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:48:59.327054 12409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "0522e2b3ecda4c0fba11b580b28c1378"
format_stamp: "Formatted at 2026-08-09 22:48:59 on dist-test-slave-chsk"
server_key: "84198a59f35d14bd4218fa4df4678751"
server_key_iv: "8ef57b7cb701725991c705d4d9ea616e"
server_key_version: "encryptionkey@0"
I20260809 22:48:59.327327 12409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:48:59.345474 12409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:48:59.349516 12409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:48:59.349803 12409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:48:59.369184 12409 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42515
I20260809 22:48:59.369220 12483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42515 every 8 connection(s)
I20260809 22:48:59.370595 12409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:48:59.372380 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12409
I20260809 22:48:59.372584 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:48:59.373050 21614 external_mini_cluster.cc:1428] Setting key ae33a073d9773e976832d067de4dad7b
I20260809 22:48:59.374940 12484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:48:59.388495 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:59.374923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:54299 (local address 127.0.0.1:42515)
0809 22:48:59.375802 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:59.375817 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:59.375864 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:48:59.378105 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:59.378132 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:59.378652 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:59.379100 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:59.379630 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:59.381677 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:59.383094 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:59.384139 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:59.384320 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:59.385622 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:59.385637 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:59.385655 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:59.386411 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:59.386505 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:59.386546 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:59.386710 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:59.386997 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:48:59.387223 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":366,"threads_started":1}
I20260809 22:48:59.389662 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378: Bootstrap starting.
I20260809 22:48:59.392228 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:48:59.392940 12484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378: Neither blocks nor log segments found. Creating new log.
I20260809 22:48:59.393913 12484 log.cc:824] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378: Log is configured to *not* fsync() on all Append() calls
I20260809 22:48:59.396986 12484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378: No bootstrap required, opened a new log
I20260809 22:48:59.403465 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:48:59.396870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:56171 (local address 127.0.0.1:42515)
0809 22:48:59.397077 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:48:59.397084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:48:59.397417 (+ 333us) server_negotiation.cc:407] Connection header received
0809 22:48:59.397569 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:48:59.397574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:48:59.397653 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:48:59.397765 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:48:59.398269 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:59.399464 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:59.400759 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:48:59.401457 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:48:59.401589 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:48:59.402645 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:48:59.402652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:48:59.402657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:48:59.402679 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:48:59.402731 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:48:59.402750 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:48:59.402853 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 22:48:59.403039 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 22:48:59.403095 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:48:59.406167 12484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } }
I20260809 22:48:59.406481 12484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:48:59.406562 12484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0522e2b3ecda4c0fba11b580b28c1378, State: Initialized, Role: FOLLOWER
I20260809 22:48:59.407001 12484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } }
I20260809 22:48:59.407282 12484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:48:59.407429 12484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:48:59.407648 12484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:48:59.409780 12484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } }
I20260809 22:48:59.410269 12484 leader_election.cc:302] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0522e2b3ecda4c0fba11b580b28c1378; no voters:
I20260809 22:48:59.410926 12484 leader_election.cc:288] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:48:59.411041 12497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:48:59.413305 12497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [term 1 LEADER]: Becoming Leader. State: Replica: 0522e2b3ecda4c0fba11b580b28c1378, State: Running, Role: LEADER
I20260809 22:48:59.413771 12484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:48:59.413949 12497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } }
I20260809 22:48:59.416927 12500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0522e2b3ecda4c0fba11b580b28c1378. Latest consensus state: current_term: 1 leader_uuid: "0522e2b3ecda4c0fba11b580b28c1378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } } }
I20260809 22:48:59.417411 12500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:59.418080 12498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0522e2b3ecda4c0fba11b580b28c1378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0522e2b3ecda4c0fba11b580b28c1378" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42515 } } }
I20260809 22:48:59.418429 12498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378 [sys.catalog]: This master's current role is: LEADER
I20260809 22:48:59.419744 12507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:48:59.427103 12507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:48:59.435577 12507 catalog_manager.cc:1347] Generated new cluster ID: 0478a39c5c994b8193cbd5e4981fa009
I20260809 22:48:59.435701 12507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:48:59.452575 12507 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:48:59.454510 12507 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:48:59.465579 12507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0522e2b3ecda4c0fba11b580b28c1378: Generated new TSK 0
I20260809 22:48:59.466523 12507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:48:59.484023 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12409
W20260809 22:48:59.485322 12479 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.110:54299
W20260809 22:48:59.486171 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12409: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:48:59.486336 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12409
W20260809 22:48:59.507047 12392 connection.cc:535] client connection to 127.0.0.1:42515 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42515): Connection reset by peer (error 104)
2026-08-09T22:48:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:48:59 dist-test-slave-chsk krb5kdc[12523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:48:59 dist-test-slave-chsk krb5kdc[12523](info): set up 2 sockets
Aug 09 22:48:59 dist-test-slave-chsk krb5kdc[12523](info): commencing operation
krb5kdc: starting...
W20260809 22:49:00.663779 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:00 dist-test-slave-chsk krb5kdc[12523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:49:00.972849 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:01.329155 12536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:01.329437 12536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:01.329527 12536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:01.336694 12536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:01.336813 12536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:01.336961 12536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:01.337077 12536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:01.337129 12536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:01.346915 12536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:01.348188 12536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:01.349424 12536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:01.361056 12546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:01.361114 12542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:01.361078 12544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:01.362332 12536 server_base.cc:1034] running on GCE node
I20260809 22:49:01.363128 12536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:01.364616 12536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:01.365865 12536 hybrid_clock.cc:648] HybridClock initialized: now 1786315741365818 us; error 48 us; skew 500 ppm
I20260809 22:49:01.366243 12536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:01 dist-test-slave-chsk krb5kdc[12523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315741, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:01.369597 12536 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:49:01.371320 12536 webserver.cc:460] Webserver started at http://127.0.0.1:46855/ using document root <none> and password file <none>
I20260809 22:49:01.372018 12536 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:01.372180 12536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:01.372498 12536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:01.375548 12536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "bbf36c4f0c704a4eadb7ec8c1770229c"
format_stamp: "Formatted at 2026-08-09 22:49:01 on dist-test-slave-chsk"
server_key: "704ce994ca65d208da3df4a16f4177a5"
server_key_iv: "8d73ab6c1a68cf62234d26b84c7d2e27"
server_key_version: "encryptionkey@0"
I20260809 22:49:01.376358 12536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "bbf36c4f0c704a4eadb7ec8c1770229c"
format_stamp: "Formatted at 2026-08-09 22:49:01 on dist-test-slave-chsk"
server_key: "704ce994ca65d208da3df4a16f4177a5"
server_key_iv: "8d73ab6c1a68cf62234d26b84c7d2e27"
server_key_version: "encryptionkey@0"
I20260809 22:49:01.381669 12536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:49:01.385052 12554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:01.386314 12536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:49:01.386499 12536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "bbf36c4f0c704a4eadb7ec8c1770229c"
format_stamp: "Formatted at 2026-08-09 22:49:01 on dist-test-slave-chsk"
server_key: "704ce994ca65d208da3df4a16f4177a5"
server_key_iv: "8d73ab6c1a68cf62234d26b84c7d2e27"
server_key_version: "encryptionkey@0"
I20260809 22:49:01.386699 12536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:01.392582 12536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:01.396644 12536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:01.396957 12536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:01.415900 12536 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38641
I20260809 22:49:01.415948 12613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38641 every 8 connection(s)
I20260809 22:49:01.417292 12536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:01.421406 12614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:01.423928 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12536
I20260809 22:49:01.424126 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:01.424674 21614 external_mini_cluster.cc:1428] Setting key 5a66c3bee04ff822f017de8b456b5d8f
I20260809 22:49:01.432444 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:01.426479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:33407 (local address 127.0.0.1:38641)
0809 22:49:01.427354 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:01.427366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:01.427411 (+ 45us) server_negotiation.cc:407] Connection header received
0809 22:49:01.429071 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:01.429092 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:01.429400 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:01.429733 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:01.430832 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:01.430838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:01.430853 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:01.430867 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:01.430927 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:01.430952 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:01.431124 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:01.431307 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 22:49:01.431548 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":396,"threads_started":1}
I20260809 22:49:01.433975 12614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c: Bootstrap starting.
I20260809 22:49:01.436018 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:01.437470 12614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:01.438743 12614 log.cc:824] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:01.441707 12614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c: No bootstrap required, opened a new log
I20260809 22:49:01.444198 12617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:01.441651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:44269 (local address 127.0.0.1:38641)
0809 22:49:01.441831 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:01.441838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:01.442197 (+ 359us) server_negotiation.cc:407] Connection header received
0809 22:49:01.442332 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:01.442337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:01.442399 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:01.442510 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:01.443484 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:01.443490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:01.443495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:01.443514 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:01.443564 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:01.443589 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:01.443706 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:01.443820 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 22:49:01.443889 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:49:01.450095 12614 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } }
I20260809 22:49:01.450492 12614 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:01.450619 12614 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbf36c4f0c704a4eadb7ec8c1770229c, State: Initialized, Role: FOLLOWER
I20260809 22:49:01.451233 12614 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } }
I20260809 22:49:01.451474 12614 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:01.451632 12614 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:01.451795 12614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:01.453986 12614 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } }
I20260809 22:49:01.454509 12614 leader_election.cc:302] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbf36c4f0c704a4eadb7ec8c1770229c; no voters:
I20260809 22:49:01.455190 12614 leader_election.cc:288] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:01.455370 12624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:01.456180 12624 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [term 1 LEADER]: Becoming Leader. State: Replica: bbf36c4f0c704a4eadb7ec8c1770229c, State: Running, Role: LEADER
I20260809 22:49:01.456779 12624 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } }
I20260809 22:49:01.457396 12614 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:01.460043 12627 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbf36c4f0c704a4eadb7ec8c1770229c. Latest consensus state: current_term: 1 leader_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } } }
I20260809 22:49:01.460534 12627 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:01.460350 12626 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbf36c4f0c704a4eadb7ec8c1770229c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38641 } } }
I20260809 22:49:01.461046 12626 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:01.463006 12634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:01.468321 12634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:01.477780 12634 catalog_manager.cc:1347] Generated new cluster ID: 8df02a7fd1a649558913ff78fdbe3b40
I20260809 22:49:01.477931 12634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:01.489288 12634 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:01.489737 12634 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:01.498966 12634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbf36c4f0c704a4eadb7ec8c1770229c: Generated new TSK 0
I20260809 22:49:01.499809 12634 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:01.505663 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12536
W20260809 22:49:01.507068 12608 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.110:33407
W20260809 22:49:01.507876 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12536: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:01.508040 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12536
2026-08-09T22:49:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:01 dist-test-slave-chsk krb5kdc[12648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:01 dist-test-slave-chsk krb5kdc[12648](info): set up 2 sockets
Aug 09 22:49:01 dist-test-slave-chsk krb5kdc[12648](info): commencing operation
krb5kdc: starting...
W20260809 22:49:02.687512 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.009s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:02 dist-test-slave-chsk krb5kdc[12648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:49:03.007530 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:03.340090 12663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:03.340376 12663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:03.340571 12663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:03.347868 12663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:03.347975 12663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:03.348060 12663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:03.348176 12663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:03.348258 12663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:03.357869 12663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:03.359112 12663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:03.360366 12663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:03.372113 12670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:03.372185 12671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:03.372138 12674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:03.373327 12663 server_base.cc:1034] running on GCE node
I20260809 22:49:03.374321 12663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:03.375837 12663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:03.377067 12663 hybrid_clock.cc:648] HybridClock initialized: now 1786315743377009 us; error 47 us; skew 500 ppm
I20260809 22:49:03.377420 12663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:03 dist-test-slave-chsk krb5kdc[12648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:03.380579 12663 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:49:03.382198 12663 webserver.cc:460] Webserver started at http://127.0.0.1:39247/ using document root <none> and password file <none>
I20260809 22:49:03.382864 12663 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:03.383015 12663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:03.383387 12663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:03.386339 12663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "fbaa5e45c3774b299d41f00b81e3b801"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-chsk"
server_key: "dc9841f87db05609802494950e9a1188"
server_key_iv: "6059ef182f0033ce304f78aa474f3da2"
server_key_version: "encryptionkey@0"
I20260809 22:49:03.387113 12663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "fbaa5e45c3774b299d41f00b81e3b801"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-chsk"
server_key: "dc9841f87db05609802494950e9a1188"
server_key_iv: "6059ef182f0033ce304f78aa474f3da2"
server_key_version: "encryptionkey@0"
I20260809 22:49:03.392132 12663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:49:03.395463 12684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:03.396654 12663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:49:03.396847 12663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "fbaa5e45c3774b299d41f00b81e3b801"
format_stamp: "Formatted at 2026-08-09 22:49:03 on dist-test-slave-chsk"
server_key: "dc9841f87db05609802494950e9a1188"
server_key_iv: "6059ef182f0033ce304f78aa474f3da2"
server_key_version: "encryptionkey@0"
I20260809 22:49:03.397044 12663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:03.412671 12663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:03.416639 12663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:03.416926 12663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:03.437022 12663 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42187
I20260809 22:49:03.437109 12741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42187 every 8 connection(s)
I20260809 22:49:03.438414 12663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:03.442147 12742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:03.446326 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12663
I20260809 22:49:03.446524 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:03.447108 21614 external_mini_cluster.cc:1428] Setting key f6b26bd2579a7c23aa0ebebf24b03ba2
I20260809 22:49:03.453789 12742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801: Bootstrap starting.
I20260809 22:49:03.456815 12742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:03.457811 12742 log.cc:824] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:03.460623 12742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801: No bootstrap required, opened a new log
Aug 09 22:49:03 dist-test-slave-chsk krb5kdc[12648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 22:49:03.471181 12742 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } }
I20260809 22:49:03.471642 12742 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:03.471771 12742 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbaa5e45c3774b299d41f00b81e3b801, State: Initialized, Role: FOLLOWER
I20260809 22:49:03.471853 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:03.449043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:48025 (local address 127.0.0.1:42187)
0809 22:49:03.449777 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:03.449789 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:03.449826 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:49:03.452675 (+ 2849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:03.452705 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:03.453071 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:03.453445 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:03.454932 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.456301 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.457623 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.458534 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.458754 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:03.462912 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:03.462922 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:03.462939 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:03.462988 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:03.467079 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.467934 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.467948 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.467956 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.468167 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.468592 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.468599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.468607 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.469367 (+ 760us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:03.469698 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:03.470130 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 22:49:03.470478 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":313,"threads_started":1}
I20260809 22:49:03.472378 12742 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } }
I20260809 22:49:03.472621 12742 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:03.472806 12742 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:03.472988 12742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:03.476087 12742 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } }
I20260809 22:49:03.476359 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:03.476630 12742 leader_election.cc:302] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbaa5e45c3774b299d41f00b81e3b801; no voters:
I20260809 22:49:03.477264 12742 leader_election.cc:288] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:03.477461 12747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:03.479331 12742 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:03.479562 12747 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [term 1 LEADER]: Becoming Leader. State: Replica: fbaa5e45c3774b299d41f00b81e3b801, State: Running, Role: LEADER
I20260809 22:49:03.480291 12747 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } }
I20260809 22:49:03.483640 12748 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbaa5e45c3774b299d41f00b81e3b801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } } }
I20260809 22:49:03.484144 12748 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:03.484782 12749 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbaa5e45c3774b299d41f00b81e3b801. Latest consensus state: current_term: 1 leader_uuid: "fbaa5e45c3774b299d41f00b81e3b801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbaa5e45c3774b299d41f00b81e3b801" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42187 } } }
I20260809 22:49:03.485170 12749 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:03.487172 12757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:03.492674 12757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:03.505334 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:03.485053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:60641 (local address 127.0.0.1:42187)
0809 22:49:03.485235 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:03.485241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:03.486935 (+ 1694us) server_negotiation.cc:407] Connection header received
0809 22:49:03.487136 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:03.487167 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:03.487259 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:03.487386 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:03.489681 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.490928 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.492834 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:03.493563 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:03.493681 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:03.497342 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:03.497351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:03.497360 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:03.497401 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:03.501454 (+ 4053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.502572 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.502579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.502582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.502739 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:03.503374 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:03.503384 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:03.503388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:03.503779 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:03.504028 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:03.504765 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 22:49:03.504956 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:49:03.511124 12757 catalog_manager.cc:1347] Generated new cluster ID: bdf5b2eca22d4e1fa7336b241cd853b3
I20260809 22:49:03.511286 12757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:03.532512 12757 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:03.533926 12757 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:03.549507 12757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbaa5e45c3774b299d41f00b81e3b801: Generated new TSK 0
I20260809 22:49:03.550568 12757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:03.565924 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12663
I20260809 22:49:03.567595 12737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:03.661968 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12663
W20260809 22:49:03.680397 12646 connection.cc:535] client connection to 127.0.0.1:42187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42187): Connection reset by peer (error 104)
2026-08-09T22:49:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2151 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:03 dist-test-slave-chsk krb5kdc[12774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:03 dist-test-slave-chsk krb5kdc[12774](info): set up 2 sockets
Aug 09 22:49:03 dist-test-slave-chsk krb5kdc[12774](info): commencing operation
krb5kdc: starting...
W20260809 22:49:04.854422 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.007s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:05 dist-test-slave-chsk krb5kdc[12774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:49:05.176550 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:05.511792 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:05.512084 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:05.512259 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:05.519649 12789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:05.519762 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:05.519878 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:05.519992 12789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:05.520071 12789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:05.530316 12789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:05.531620 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:05.532923 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:05.544981 12796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:05.545030 12797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:05.545089 12799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:05.545785 12789 server_base.cc:1034] running on GCE node
I20260809 22:49:05.546581 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:05.548206 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:05.549454 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786315745549478 us; error 130 us; skew 500 ppm
I20260809 22:49:05.549817 12789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:05 dist-test-slave-chsk krb5kdc[12774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315745, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:05.553037 12789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:49:05.554589 12789 webserver.cc:460] Webserver started at http://127.0.0.1:33359/ using document root <none> and password file <none>
I20260809 22:49:05.555362 12789 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:05.555514 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:05.555840 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:05.558817 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "52b13b9b83ab490792732969625d9bb3"
format_stamp: "Formatted at 2026-08-09 22:49:05 on dist-test-slave-chsk"
server_key: "92b823770f2238c051364a505c79e21f"
server_key_iv: "bf7b2e69d298b1f1c64fcf44ac9ace7a"
server_key_version: "encryptionkey@0"
I20260809 22:49:05.559617 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "52b13b9b83ab490792732969625d9bb3"
format_stamp: "Formatted at 2026-08-09 22:49:05 on dist-test-slave-chsk"
server_key: "92b823770f2238c051364a505c79e21f"
server_key_iv: "bf7b2e69d298b1f1c64fcf44ac9ace7a"
server_key_version: "encryptionkey@0"
I20260809 22:49:05.564745 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:49:05.568305 12807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:05.569600 12789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:49:05.569813 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "52b13b9b83ab490792732969625d9bb3"
format_stamp: "Formatted at 2026-08-09 22:49:05 on dist-test-slave-chsk"
server_key: "92b823770f2238c051364a505c79e21f"
server_key_iv: "bf7b2e69d298b1f1c64fcf44ac9ace7a"
server_key_version: "encryptionkey@0"
I20260809 22:49:05.570016 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:05.598732 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:05.602910 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:05.603247 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:05.622869 12789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40607
I20260809 22:49:05.622916 12862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40607 every 8 connection(s)
I20260809 22:49:05.624331 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:05.625527 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12789
I20260809 22:49:05.625725 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:05.626317 21614 external_mini_cluster.cc:1428] Setting key b892095d250812ea7b1c607a7653c835
I20260809 22:49:05.629041 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:05.639803 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:05.628215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:52045 (local address 127.0.0.1:40607)
0809 22:49:05.628992 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:05.629003 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:05.629046 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:49:05.630757 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:05.630777 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:05.631187 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:05.631547 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:05.632146 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.633439 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.635390 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.636015 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.636140 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:05.637478 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:05.637484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:05.637496 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:05.638021 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:05.638094 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:05.638120 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:05.638259 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:05.638548 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 22:49:05.638751 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":361,"threads_started":1}
I20260809 22:49:05.641485 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3: Bootstrap starting.
I20260809 22:49:05.643639 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:05.644865 12863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:05.645849 12863 log.cc:824] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:05.648846 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3: No bootstrap required, opened a new log
I20260809 22:49:05.654740 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:05.648135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:48819 (local address 127.0.0.1:40607)
0809 22:49:05.648323 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:05.648328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:05.648711 (+ 383us) server_negotiation.cc:407] Connection header received
0809 22:49:05.648861 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:05.648869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:05.648952 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:05.649050 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:05.649541 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.650657 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.652004 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:05.652733 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:05.652880 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:05.653911 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:05.653923 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:05.653928 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:05.653949 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:05.654010 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:05.654030 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:05.654156 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:05.654325 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 22:49:05.654377 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:49:05.657660 12863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } }
I20260809 22:49:05.658008 12863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:05.658126 12863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b13b9b83ab490792732969625d9bb3, State: Initialized, Role: FOLLOWER
I20260809 22:49:05.658664 12863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } }
I20260809 22:49:05.658891 12863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:05.659049 12863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:05.659253 12863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:05.661418 12863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } }
I20260809 22:49:05.661907 12863 leader_election.cc:302] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52b13b9b83ab490792732969625d9bb3; no voters:
I20260809 22:49:05.662516 12863 leader_election.cc:288] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:05.662639 12874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:05.663393 12874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [term 1 LEADER]: Becoming Leader. State: Replica: 52b13b9b83ab490792732969625d9bb3, State: Running, Role: LEADER
I20260809 22:49:05.663900 12874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } }
I20260809 22:49:05.664561 12863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:05.666684 12878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52b13b9b83ab490792732969625d9bb3. Latest consensus state: current_term: 1 leader_uuid: "52b13b9b83ab490792732969625d9bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } } }
I20260809 22:49:05.666785 12876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52b13b9b83ab490792732969625d9bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52b13b9b83ab490792732969625d9bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40607 } } }
I20260809 22:49:05.667227 12876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:05.667232 12878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:05.672394 12883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:05.678733 12883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:05.687115 12883 catalog_manager.cc:1347] Generated new cluster ID: 2eb84ceca4d8422eb272ed5542c82586
I20260809 22:49:05.687244 12883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:05.720604 12883 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:05.722512 12883 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:05.745543 12883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52b13b9b83ab490792732969625d9bb3: Generated new TSK 0
I20260809 22:49:05.746479 12883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:05.754880 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12789
W20260809 22:49:05.756021 12858 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.110:52045
W20260809 22:49:05.756750 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12789: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:05.756906 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12789
W20260809 22:49:05.777407 12772 connection.cc:535] client connection to 127.0.0.1:40607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40607): Connection reset by peer (error 104)
2026-08-09T22:49:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2097 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:05 dist-test-slave-chsk krb5kdc[12897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:05 dist-test-slave-chsk krb5kdc[12897](info): set up 2 sockets
Aug 09 22:49:05 dist-test-slave-chsk krb5kdc[12897](info): commencing operation
krb5kdc: starting...
W20260809 22:49:06.935328 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:07 dist-test-slave-chsk krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 22:49:07.264019 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:07.611805 12911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:07.612124 12911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:07.612282 12911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:07.619729 12911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:07.619891 12911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:07.619984 12911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:07.620097 12911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:07.620160 12911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:07.630509 12911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:07.632365 12911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:07.633755 12911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:07.646345 12919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:07.646359 12916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:07.646384 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:07.647254 12911 server_base.cc:1034] running on GCE node
I20260809 22:49:07.648083 12911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:07.649444 12911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:07.650647 12911 hybrid_clock.cc:648] HybridClock initialized: now 1786315747650601 us; error 37 us; skew 500 ppm
I20260809 22:49:07.651003 12911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:07 dist-test-slave-chsk krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315747, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:07.657616 12911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 22:49:07.659235 12911 webserver.cc:460] Webserver started at http://127.0.0.1:38113/ using document root <none> and password file <none>
I20260809 22:49:07.659945 12911 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:07.660105 12911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:07.660426 12911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:07.663587 12911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "9feea13134da48a186c7b1ce365305c8"
format_stamp: "Formatted at 2026-08-09 22:49:07 on dist-test-slave-chsk"
server_key: "474a3c692f2b67541cd07ccd682af53d"
server_key_iv: "f8f9972b3fe5e39dd1b21b4c504b1bdf"
server_key_version: "encryptionkey@0"
I20260809 22:49:07.664361 12911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "9feea13134da48a186c7b1ce365305c8"
format_stamp: "Formatted at 2026-08-09 22:49:07 on dist-test-slave-chsk"
server_key: "474a3c692f2b67541cd07ccd682af53d"
server_key_iv: "f8f9972b3fe5e39dd1b21b4c504b1bdf"
server_key_version: "encryptionkey@0"
I20260809 22:49:07.669616 12911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:07.673067 12926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:07.674398 12911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:07.674606 12911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "9feea13134da48a186c7b1ce365305c8"
format_stamp: "Formatted at 2026-08-09 22:49:07 on dist-test-slave-chsk"
server_key: "474a3c692f2b67541cd07ccd682af53d"
server_key_iv: "f8f9972b3fe5e39dd1b21b4c504b1bdf"
server_key_version: "encryptionkey@0"
I20260809 22:49:07.674887 12911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:07.680538 12911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:07.684980 12911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:07.685353 12911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:07.704404 12911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36041
I20260809 22:49:07.704480 12984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36041 every 8 connection(s)
I20260809 22:49:07.705858 12911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:07.709518 12985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:07.715742 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 12911
I20260809 22:49:07.715935 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:07.716512 21614 external_mini_cluster.cc:1428] Setting key 6d60164305014d7e36fa56e74200df17
I20260809 22:49:07.720589 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8: Bootstrap starting.
I20260809 22:49:07.724262 12985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:07.724517 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:07.718380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:58299 (local address 127.0.0.1:36041)
0809 22:49:07.719286 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:07.719297 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:07.719338 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:49:07.721048 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:07.721072 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:07.721389 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:07.721720 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:07.722853 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:07.722860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:07.722871 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:07.722885 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:07.722938 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:07.722958 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:07.723121 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:07.723354 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 22:49:07.723591 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":407,"threads_started":1}
I20260809 22:49:07.725418 12985 log.cc:824] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:07.728441 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:07.728758 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8: No bootstrap required, opened a new log
I20260809 22:49:07.741071 12985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } }
I20260809 22:49:07.741603 12985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:07.741706 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:07.737851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.110:37581 (local address 127.0.0.1:36041)
0809 22:49:07.738057 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:07.738068 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:07.739663 (+ 1595us) server_negotiation.cc:407] Connection header received
0809 22:49:07.739814 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:07.739820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:07.739887 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:07.740011 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:07.741022 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:07.741028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:07.741033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:07.741059 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:07.741117 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:07.741139 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:07.741268 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:07.741351 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 22:49:07.741414 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 22:49:07.741710 12985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9feea13134da48a186c7b1ce365305c8, State: Initialized, Role: FOLLOWER
I20260809 22:49:07.742460 12985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } }
I20260809 22:49:07.742785 12985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:07.742939 12985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:07.743170 12985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:07.745428 12985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } }
I20260809 22:49:07.745995 12985 leader_election.cc:302] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9feea13134da48a186c7b1ce365305c8; no voters:
I20260809 22:49:07.746606 12985 leader_election.cc:288] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:07.746740 12995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:07.747839 12995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [term 1 LEADER]: Becoming Leader. State: Replica: 9feea13134da48a186c7b1ce365305c8, State: Running, Role: LEADER
I20260809 22:49:07.748369 12995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } }
I20260809 22:49:07.749101 12985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:07.751372 12996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9feea13134da48a186c7b1ce365305c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } } }
I20260809 22:49:07.751772 12996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:07.752331 12997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9feea13134da48a186c7b1ce365305c8. Latest consensus state: current_term: 1 leader_uuid: "9feea13134da48a186c7b1ce365305c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9feea13134da48a186c7b1ce365305c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36041 } } }
I20260809 22:49:07.752696 12997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:07.753681 13003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:07.759979 13003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:07.769949 13003 catalog_manager.cc:1347] Generated new cluster ID: 42966be3b251460dabc1856ef7d1585c
I20260809 22:49:07.770056 13003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:07.791615 13003 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:07.792122 13003 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:07.805518 13003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9feea13134da48a186c7b1ce365305c8: Generated new TSK 0
I20260809 22:49:07.806540 13003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:07.814252 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12911
W20260809 22:49:07.815503 12980 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.110:58299
W20260809 22:49:07.816259 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 12911: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:07.816417 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 12911
2026-08-09T22:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2058 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:07 dist-test-slave-chsk krb5kdc[13017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:07 dist-test-slave-chsk krb5kdc[13017](info): set up 2 sockets
Aug 09 22:49:07 dist-test-slave-chsk krb5kdc[13017](info): commencing operation
krb5kdc: starting...
W20260809 22:49:08.984285 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:09 dist-test-slave-chsk krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:09.286135 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38519
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:38519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:09.626205 13031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:09.626519 13031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:09.626619 13031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:09.634096 13031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:09.634253 13031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:09.634303 13031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:09.634359 13031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:09.634404 13031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:09.644670 13031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:38519
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:38519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:09.645988 13031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:09.647328 13031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:09.659089 13037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.659133 13040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:09.659116 13038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:09.660135 13031 server_base.cc:1034] running on GCE node
I20260809 22:49:09.660869 13031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:09.662289 13031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:09.663498 13031 hybrid_clock.cc:648] HybridClock initialized: now 1786315749663441 us; error 37 us; skew 500 ppm
I20260809 22:49:09.663869 13031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:09 dist-test-slave-chsk krb5kdc[13017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:09.667001 13031 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:09.668771 13031 webserver.cc:460] Webserver started at http://127.21.27.190:43993/ using document root <none> and password file <none>
I20260809 22:49:09.669471 13031 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:09.669605 13031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:09.669930 13031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:09.672982 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "a9ec9d6080084889b73d36cfa6cf534a"
format_stamp: "Formatted at 2026-08-09 22:49:09 on dist-test-slave-chsk"
server_key: "7169628cd7a8043013ade363d6492737"
server_key_iv: "64c0563935586dad9173eb03745aa569"
server_key_version: "encryptionkey@0"
I20260809 22:49:09.673769 13031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "a9ec9d6080084889b73d36cfa6cf534a"
format_stamp: "Formatted at 2026-08-09 22:49:09 on dist-test-slave-chsk"
server_key: "7169628cd7a8043013ade363d6492737"
server_key_iv: "64c0563935586dad9173eb03745aa569"
server_key_version: "encryptionkey@0"
I20260809 22:49:09.679107 13031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:49:09.682650 13048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:09.683974 13031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:49:09.684180 13031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "a9ec9d6080084889b73d36cfa6cf534a"
format_stamp: "Formatted at 2026-08-09 22:49:09 on dist-test-slave-chsk"
server_key: "7169628cd7a8043013ade363d6492737"
server_key_iv: "64c0563935586dad9173eb03745aa569"
server_key_version: "encryptionkey@0"
I20260809 22:49:09.684386 13031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:09.723676 13031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:09.728088 13031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:09.728403 13031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:09.751258 13031 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:38519
I20260809 22:49:09.751307 13104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:38519 every 8 connection(s)
I20260809 22:49:09.752712 13031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:09.756417 13105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:09.756641 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13031
I20260809 22:49:09.756850 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:09.757365 21614 external_mini_cluster.cc:1428] Setting key 5b4348a6fd822e1a3987c949fc630d1d
I20260809 22:49:09.770488 13105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a: Bootstrap starting.
Aug 09 22:49:09 dist-test-slave-chsk krb5kdc[13017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:09.774329 13105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:09.775516 13105 log.cc:824] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:09.778851 13105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a: No bootstrap required, opened a new log
I20260809 22:49:09.782856 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:09.759241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58371 (local address 127.21.27.190:38519)
0809 22:49:09.760059 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:09.760070 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:09.760111 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:49:09.762507 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:09.762529 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:09.762931 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:09.763339 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:09.765118 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.766441 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.768329 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.769009 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.769158 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:09.774089 (+ 4931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:09.774097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:09.774111 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:09.774166 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:09.777973 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.778839 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.778848 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.778856 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.779055 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.779578 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.779583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.779586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.780276 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:09.780583 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:09.781092 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 22:49:09.781433 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":395,"threads_started":1}
I20260809 22:49:09.787202 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:09.789307 13105 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } }
I20260809 22:49:09.789748 13105 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:09.789870 13105 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ec9d6080084889b73d36cfa6cf534a, State: Initialized, Role: FOLLOWER
I20260809 22:49:09.790544 13105 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } }
I20260809 22:49:09.790874 13105 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:09.791028 13105 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:09.791282 13105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:09.794291 13105 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } }
I20260809 22:49:09.794803 13105 leader_election.cc:302] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9ec9d6080084889b73d36cfa6cf534a; no voters:
I20260809 22:49:09.795466 13105 leader_election.cc:288] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:09.795565 13117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:09.797456 13117 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [term 1 LEADER]: Becoming Leader. State: Replica: a9ec9d6080084889b73d36cfa6cf534a, State: Running, Role: LEADER
I20260809 22:49:09.797971 13105 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:09.798233 13117 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } }
I20260809 22:49:09.805418 13121 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9ec9d6080084889b73d36cfa6cf534a. Latest consensus state: current_term: 1 leader_uuid: "a9ec9d6080084889b73d36cfa6cf534a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } } }
I20260809 22:49:09.805938 13121 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:09.806555 13120 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9ec9d6080084889b73d36cfa6cf534a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9ec9d6080084889b73d36cfa6cf534a" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 38519 } } }
I20260809 22:49:09.807260 13120 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:09.808120 13128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:09.813139 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:09.792854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33299 (local address 127.21.27.190:38519)
0809 22:49:09.793058 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:09.793065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:09.793589 (+ 524us) server_negotiation.cc:407] Connection header received
0809 22:49:09.793741 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:09.793747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:09.793828 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:09.793932 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:09.795693 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.797085 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.798960 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:09.799893 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:09.800051 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:09.804444 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:09.804449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:09.804452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:09.804489 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:09.808380 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.809529 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.809535 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.809547 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.809706 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:09.810354 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:09.810359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:09.810362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:09.810902 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:09.811204 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:09.812573 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0809 22:49:09.812753 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:49:09.813521 13128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:09.822116 13128 catalog_manager.cc:1347] Generated new cluster ID: 3e4013a2d1b84eac8b4ec117af6636a5
I20260809 22:49:09.822254 13128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:09.834649 13128 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:09.836046 13128 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:09.841588 13128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9ec9d6080084889b73d36cfa6cf534a: Generated new TSK 0
I20260809 22:49:09.842618 13128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:09.855249 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13031
I20260809 22:49:09.856437 13099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:09.947252 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13031
W20260809 22:49:09.968379 13015 connection.cc:535] client connection to 127.21.27.190:38519 recv error: Network error: failed to read from TLS socket (remote: 127.21.27.190:38519): Connection reset by peer (error 104)
2026-08-09T22:49:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2131 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:10 dist-test-slave-chsk krb5kdc[13143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:10 dist-test-slave-chsk krb5kdc[13143](info): set up 2 sockets
Aug 09 22:49:10 dist-test-slave-chsk krb5kdc[13143](info): commencing operation
krb5kdc: starting...
W20260809 22:49:11.125835 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:11 dist-test-slave-chsk krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:11.474716 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45279
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:35105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:45279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:11.825559 13157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:11.825882 13157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:11.825977 13157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:11.833434 13157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:11.833562 13157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:11.833611 13157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:11.833714 13157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:11.833797 13157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:11.844018 13157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:35105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:45279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:45279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:11.845409 13157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:11.846823 13157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:11.859181 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:11.859210 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:11.859246 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:11.860140 13157 server_base.cc:1034] running on GCE node
I20260809 22:49:11.860946 13157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:11.862511 13157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:11.863714 13157 hybrid_clock.cc:648] HybridClock initialized: now 1786315751863675 us; error 38 us; skew 500 ppm
I20260809 22:49:11.864077 13157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:11 dist-test-slave-chsk krb5kdc[13143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:11.867363 13157 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:11.868978 13157 webserver.cc:460] Webserver started at http://127.21.27.190:39405/ using document root <none> and password file <none>
I20260809 22:49:11.869690 13157 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:11.869844 13157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:11.870172 13157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:11.873400 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "e7fee9b3949f4e86a1c986b0209286a9"
format_stamp: "Formatted at 2026-08-09 22:49:11 on dist-test-slave-chsk"
server_key: "49c510b4dacc1864f6339df591461229"
server_key_iv: "fae529bb55d74f16f3bc79c803a941a6"
server_key_version: "encryptionkey@0"
I20260809 22:49:11.874213 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "e7fee9b3949f4e86a1c986b0209286a9"
format_stamp: "Formatted at 2026-08-09 22:49:11 on dist-test-slave-chsk"
server_key: "49c510b4dacc1864f6339df591461229"
server_key_iv: "fae529bb55d74f16f3bc79c803a941a6"
server_key_version: "encryptionkey@0"
I20260809 22:49:11.880730 13157 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 22:49:11.885991 13173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:11.887916 13157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 22:49:11.888152 13157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "e7fee9b3949f4e86a1c986b0209286a9"
format_stamp: "Formatted at 2026-08-09 22:49:11 on dist-test-slave-chsk"
server_key: "49c510b4dacc1864f6339df591461229"
server_key_iv: "fae529bb55d74f16f3bc79c803a941a6"
server_key_version: "encryptionkey@0"
I20260809 22:49:11.888422 13157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:11.903362 13157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:11.907966 13157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:11.908221 13157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:11.927806 13157 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:45279
I20260809 22:49:11.927845 13230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:45279 every 8 connection(s)
I20260809 22:49:11.929250 13157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:11.932957 13231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:11.938813 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13157
I20260809 22:49:11.939009 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:11.939527 21614 external_mini_cluster.cc:1428] Setting key 63ef3a9ef0e6324edc19b7dfbb6c3803
I20260809 22:49:11.944790 13231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9: Bootstrap starting.
I20260809 22:49:11.948794 13231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:11.949910 13231 log.cc:824] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:11.952898 13231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9: No bootstrap required, opened a new log
I20260809 22:49:11.954677 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.941422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53369 (local address 127.21.27.190:45279)
0809 22:49:11.942329 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:11.942346 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:11.942399 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:49:11.944803 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:11.944832 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:11.945331 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:11.945744 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:11.946229 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.948306 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.949719 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.950794 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.951014 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:11.952388 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:11.952395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:11.952409 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:11.952937 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:11.953040 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:11.953074 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:11.953256 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:11.953545 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 22:49:11.953722 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":394,"threads_started":1}
I20260809 22:49:11.959024 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:11.961843 13231 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } }
I20260809 22:49:11.962311 13231 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:11.962431 13231 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7fee9b3949f4e86a1c986b0209286a9, State: Initialized, Role: FOLLOWER
I20260809 22:49:11.963016 13231 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } }
I20260809 22:49:11.963320 13231 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:11.963482 13231 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:11.963680 13231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:11.966567 13231 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } }
I20260809 22:49:11.967094 13231 leader_election.cc:302] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7fee9b3949f4e86a1c986b0209286a9; no voters:
I20260809 22:49:11.967779 13231 leader_election.cc:288] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:11.967939 13243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:11.969656 13243 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [term 1 LEADER]: Becoming Leader. State: Replica: e7fee9b3949f4e86a1c986b0209286a9, State: Running, Role: LEADER
I20260809 22:49:11.970119 13231 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:11.970353 13243 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } }
I20260809 22:49:11.973219 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:11.964665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44357 (local address 127.21.27.190:45279)
0809 22:49:11.964848 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:11.964852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:11.965322 (+ 470us) server_negotiation.cc:407] Connection header received
0809 22:49:11.965542 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:11.965547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:11.965626 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:11.965742 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:11.966250 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.967072 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.969374 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:11.970124 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:11.970279 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:11.971734 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:11.971740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:11.971745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:11.971766 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:11.971849 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:11.971877 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:11.972026 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:11.972639 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 22:49:11.972718 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:49:11.977195 13244 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7fee9b3949f4e86a1c986b0209286a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } } }
I20260809 22:49:11.977677 13244 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:11.978157 13245 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7fee9b3949f4e86a1c986b0209286a9. Latest consensus state: current_term: 1 leader_uuid: "e7fee9b3949f4e86a1c986b0209286a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7fee9b3949f4e86a1c986b0209286a9" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 45279 } } }
I20260809 22:49:11.978562 13245 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:11.981869 13257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:11.986624 13257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:11.996182 13257 catalog_manager.cc:1347] Generated new cluster ID: c7dca2d5f5834df08c35e6f65b17a8c0
I20260809 22:49:11.996323 13257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:12.023981 13257 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:12.025961 13257 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:12.035773 13257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7fee9b3949f4e86a1c986b0209286a9: Generated new TSK 0
I20260809 22:49:12.036617 13257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:12.046283 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13157
W20260809 22:49:12.047547 13225 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53369
W20260809 22:49:12.048362 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:12.048524 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13157
W20260809 22:49:12.069250 13141 connection.cc:535] client connection to 127.21.27.190:45279 recv error: Network error: failed to read from TLS socket (remote: 127.21.27.190:45279): Connection reset by peer (error 104)
2026-08-09T22:49:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2100 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:12 dist-test-slave-chsk krb5kdc[13267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:12 dist-test-slave-chsk krb5kdc[13267](info): set up 2 sockets
Aug 09 22:49:12 dist-test-slave-chsk krb5kdc[13267](info): commencing operation
krb5kdc: starting...
W20260809 22:49:13.232702 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:13 dist-test-slave-chsk krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:13.532558 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37277
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:37277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:13.865679 13280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:13.865954 13280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:13.866047 13280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:13.873291 13280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:13.873407 13280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:13.873454 13280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:13.873515 13280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:13.873553 13280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:13.883214 13280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:37277
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:37277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:13.884442 13280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:13.885682 13280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:13.897650 13285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:13.897627 13286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:13.897794 13288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:13.898504 13280 server_base.cc:1034] running on GCE node
I20260809 22:49:13.899366 13280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:13.900770 13280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:13.902019 13280 hybrid_clock.cc:648] HybridClock initialized: now 1786315753901963 us; error 34 us; skew 500 ppm
I20260809 22:49:13.902382 13280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:13 dist-test-slave-chsk krb5kdc[13267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315753, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:13.910542 13280 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:13.912261 13280 webserver.cc:460] Webserver started at http://127.21.27.190:44927/ using document root <none> and password file <none>
I20260809 22:49:13.912974 13280 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:13.913134 13280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:13.913460 13280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:13.916625 13280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "f851b684dee043a0a5ba30e3cd295d79"
format_stamp: "Formatted at 2026-08-09 22:49:13 on dist-test-slave-chsk"
server_key: "b49959f4397a6699de17fac7ee4df570"
server_key_iv: "ec7d6aa63b11502121e919dcbe36dd5f"
server_key_version: "encryptionkey@0"
I20260809 22:49:13.917411 13280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "f851b684dee043a0a5ba30e3cd295d79"
format_stamp: "Formatted at 2026-08-09 22:49:13 on dist-test-slave-chsk"
server_key: "b49959f4397a6699de17fac7ee4df570"
server_key_iv: "ec7d6aa63b11502121e919dcbe36dd5f"
server_key_version: "encryptionkey@0"
I20260809 22:49:13.922654 13280 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:49:13.926088 13295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:13.927389 13280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:13.927585 13280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "f851b684dee043a0a5ba30e3cd295d79"
format_stamp: "Formatted at 2026-08-09 22:49:13 on dist-test-slave-chsk"
server_key: "b49959f4397a6699de17fac7ee4df570"
server_key_iv: "ec7d6aa63b11502121e919dcbe36dd5f"
server_key_version: "encryptionkey@0"
I20260809 22:49:13.927796 13280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:13.933612 13280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:13.937645 13280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:13.937954 13280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:13.956861 13280 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:37277
I20260809 22:49:13.956895 13356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:37277 every 8 connection(s)
I20260809 22:49:13.958220 13280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:13.961750 13357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:13.963376 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13280
I20260809 22:49:13.963565 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:13.964004 21614 external_mini_cluster.cc:1428] Setting key 9eb373de13504cb3f43dd0edc467df5a
I20260809 22:49:13.971560 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:13.965792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58909 (local address 127.21.27.190:37277)
0809 22:49:13.966642 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:13.966652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:13.966695 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:49:13.968350 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:13.968374 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:13.968687 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:13.968993 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:13.970041 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:13.970047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:13.970058 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:13.970072 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:13.970127 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:13.970145 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:13.970325 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:13.970423 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 22:49:13.970605 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":402,"threads_started":1}
I20260809 22:49:13.974628 13357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79: Bootstrap starting.
I20260809 22:49:13.975072 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:13.977507 13357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:13.978669 13357 log.cc:824] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:13.981891 13357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79: No bootstrap required, opened a new log
I20260809 22:49:13.982005 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:13.979545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56449 (local address 127.21.27.190:37277)
0809 22:49:13.979755 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:13.979760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:13.980159 (+ 399us) server_negotiation.cc:407] Connection header received
0809 22:49:13.980295 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:13.980299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:13.980353 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:13.980461 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:13.981327 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:13.981333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:13.981337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:13.981355 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:13.981402 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:13.981425 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:13.981544 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:13.981652 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 22:49:13.981707 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:49:13.989342 13357 raft_consensus.cc:348] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } }
I20260809 22:49:13.989663 13357 raft_consensus.cc:374] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:13.989743 13357 raft_consensus.cc:729] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f851b684dee043a0a5ba30e3cd295d79, State: Initialized, Role: FOLLOWER
I20260809 22:49:13.990396 13357 consensus_queue.cc:260] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } }
I20260809 22:49:13.990680 13357 raft_consensus.cc:388] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:13.990845 13357 raft_consensus.cc:482] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:13.991020 13357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:13.993142 13357 raft_consensus.cc:504] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } }
I20260809 22:49:13.993614 13357 leader_election.cc:302] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f851b684dee043a0a5ba30e3cd295d79; no voters:
I20260809 22:49:13.994199 13357 leader_election.cc:288] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:13.994310 13368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:13.995375 13368 raft_consensus.cc:686] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [term 1 LEADER]: Becoming Leader. State: Replica: f851b684dee043a0a5ba30e3cd295d79, State: Running, Role: LEADER
I20260809 22:49:13.995944 13368 consensus_queue.cc:237] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } }
I20260809 22:49:13.996412 13357 sys_catalog.cc:564] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:13.998888 13369 sys_catalog.cc:455] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f851b684dee043a0a5ba30e3cd295d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } } }
I20260809 22:49:13.999378 13369 sys_catalog.cc:458] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:13.999014 13371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f851b684dee043a0a5ba30e3cd295d79. Latest consensus state: current_term: 1 leader_uuid: "f851b684dee043a0a5ba30e3cd295d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f851b684dee043a0a5ba30e3cd295d79" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 37277 } } }
I20260809 22:49:13.999558 13371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:14.001761 13376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:14.007122 13376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:14.016269 13376 catalog_manager.cc:1347] Generated new cluster ID: 5b3b5d518dd245d98d064604780c3dfa
I20260809 22:49:14.016397 13376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:14.040969 13376 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:14.041393 13376 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:14.052541 13376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f851b684dee043a0a5ba30e3cd295d79: Generated new TSK 0
I20260809 22:49:14.053522 13376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:14.068496 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13280
W20260809 22:49:14.069597 13350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58909
W20260809 22:49:14.070402 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13280: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:14.070590 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13280
2026-08-09T22:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2018 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:14 dist-test-slave-chsk krb5kdc[13393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:14 dist-test-slave-chsk krb5kdc[13393](info): set up 2 sockets
Aug 09 22:49:14 dist-test-slave-chsk krb5kdc[13393](info): commencing operation
krb5kdc: starting...
W20260809 22:49:15.231848 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:15 dist-test-slave-chsk krb5kdc[13393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:15.550561 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:33601
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:15.888934 13408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:15.889241 13408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:15.889401 13408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:15.896704 13408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:15.896823 13408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:15.896929 13408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:15.897055 13408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:15.897140 13408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:15.907155 13408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:15.908521 13408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:15.909825 13408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:15.922076 13414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:15.922053 13416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:15.922046 13413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:15.923250 13408 server_base.cc:1034] running on GCE node
I20260809 22:49:15.924127 13408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:15.925566 13408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:15.926803 13408 hybrid_clock.cc:648] HybridClock initialized: now 1786315755926771 us; error 44 us; skew 500 ppm
I20260809 22:49:15.927270 13408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:15 dist-test-slave-chsk krb5kdc[13393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:15.930392 13408 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:15.932013 13408 webserver.cc:460] Webserver started at http://127.21.27.190:35281/ using document root <none> and password file <none>
I20260809 22:49:15.932721 13408 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:15.932871 13408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:15.933185 13408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:15.936357 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "51523a4d62e94993a78245e5f190a0b7"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-chsk"
server_key: "9848798b816e88d50836f8ecfcf9cfff"
server_key_iv: "941480940d9e6c95b23c1d1c0f43e3cd"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.937143 13408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "51523a4d62e94993a78245e5f190a0b7"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-chsk"
server_key: "9848798b816e88d50836f8ecfcf9cfff"
server_key_iv: "941480940d9e6c95b23c1d1c0f43e3cd"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.942378 13408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:49:15.945791 13423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:15.947038 13408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:15.947252 13408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "51523a4d62e94993a78245e5f190a0b7"
format_stamp: "Formatted at 2026-08-09 22:49:15 on dist-test-slave-chsk"
server_key: "9848798b816e88d50836f8ecfcf9cfff"
server_key_iv: "941480940d9e6c95b23c1d1c0f43e3cd"
server_key_version: "encryptionkey@0"
I20260809 22:49:15.947449 13408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:15.959221 13408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:15.963052 13408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:15.963353 13408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:15.982322 13408 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:33601
I20260809 22:49:15.982359 13480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:33601 every 8 connection(s)
I20260809 22:49:15.983721 13408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:15.987548 13482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:15.991633 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13408
I20260809 22:49:15.991819 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:15.992403 21614 external_mini_cluster.cc:1428] Setting key b26253a1ab44a2ff221cd2c6d6d3e5d5
I20260809 22:49:16.001036 13482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7: Bootstrap starting.
I20260809 22:49:16.004989 13482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:16.006372 13482 log.cc:824] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:49:16 dist-test-slave-chsk krb5kdc[13393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:16.009531 13482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7: No bootstrap required, opened a new log
I20260809 22:49:16.016564 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:15.994331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40567 (local address 127.21.27.190:33601)
0809 22:49:15.995114 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:15.995126 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:15.995224 (+ 98us) server_negotiation.cc:407] Connection header received
0809 22:49:15.996896 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:15.996917 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:15.997320 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:15.997663 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:15.999556 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:16.000835 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:16.002669 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:16.003416 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:16.003552 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:16.008872 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:16.008881 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:16.008894 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:16.008933 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:16.012260 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:16.013094 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:16.013101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:16.013106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:16.013279 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:16.013682 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:16.013691 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:16.013694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:16.014257 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:16.014588 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:16.015025 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 22:49:16.015496 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":396,"threads_started":1}
I20260809 22:49:16.017722 13482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } }
I20260809 22:49:16.018112 13482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:16.018252 13482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51523a4d62e94993a78245e5f190a0b7, State: Initialized, Role: FOLLOWER
I20260809 22:49:16.018810 13482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } }
I20260809 22:49:16.019073 13482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:16.019332 13482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:16.019523 13482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:16.020661 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:16.022675 13482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } }
I20260809 22:49:16.023339 13482 leader_election.cc:302] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51523a4d62e94993a78245e5f190a0b7; no voters:
I20260809 22:49:16.024008 13482 leader_election.cc:288] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:16.024164 13487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:16.025575 13487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [term 1 LEADER]: Becoming Leader. State: Replica: 51523a4d62e94993a78245e5f190a0b7, State: Running, Role: LEADER
I20260809 22:49:16.026243 13487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } }
I20260809 22:49:16.026896 13482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:16.029683 13491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51523a4d62e94993a78245e5f190a0b7. Latest consensus state: current_term: 1 leader_uuid: "51523a4d62e94993a78245e5f190a0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } } }
I20260809 22:49:16.029963 13491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:16.029173 13490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51523a4d62e94993a78245e5f190a0b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51523a4d62e94993a78245e5f190a0b7" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 33601 } } }
I20260809 22:49:16.030321 13490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:16.031109 13497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:16.051491 13497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:16.064409 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:16.031575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42355 (local address 127.21.27.190:33601)
0809 22:49:16.031809 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:16.031818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:16.041870 (+ 10052us) server_negotiation.cc:407] Connection header received
0809 22:49:16.042537 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:16.042546 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:16.042660 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:16.042765 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:16.046211 (+ 3446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:16.047712 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:16.050277 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:16.051311 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:16.051478 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:16.055851 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:16.055861 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:16.055869 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:16.055928 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:16.060348 (+ 4420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:16.061280 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:16.061286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:16.061290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:16.061437 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:16.061882 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:16.061888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:16.061891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:16.062344 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:16.062559 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:16.063788 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0809 22:49:16.064016 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:49:16.070133 13497 catalog_manager.cc:1347] Generated new cluster ID: 7ac3112acd714726b6837db713a91d91
I20260809 22:49:16.070261 13497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:16.090265 13497 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:16.092141 13497 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:16.101346 13497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51523a4d62e94993a78245e5f190a0b7: Generated new TSK 0
I20260809 22:49:16.102188 13497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:16.108698 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13408
I20260809 22:49:16.109935 13476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:16.204723 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13408
2026-08-09T22:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:16 dist-test-slave-chsk krb5kdc[13516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:16 dist-test-slave-chsk krb5kdc[13516](info): set up 2 sockets
Aug 09 22:49:16 dist-test-slave-chsk krb5kdc[13516](info): commencing operation
krb5kdc: starting...
W20260809 22:49:17.383986 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:17 dist-test-slave-chsk krb5kdc[13516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:17.702766 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42051
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:42051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:18.045086 13531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:18.045441 13531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:18.045601 13531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:18.052918 13531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:18.053081 13531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:18.053181 13531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:18.053285 13531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:18.053365 13531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:18.063560 13531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:42051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:42051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:18.064872 13531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:18.066202 13531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:18.078378 13536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:18.078338 13539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:18.078351 13537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:18.079275 13531 server_base.cc:1034] running on GCE node
I20260809 22:49:18.080026 13531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:18.081383 13531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:18.082621 13531 hybrid_clock.cc:648] HybridClock initialized: now 1786315758082598 us; error 57 us; skew 500 ppm
I20260809 22:49:18.082962 13531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:18 dist-test-slave-chsk krb5kdc[13516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315758, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:18.086103 13531 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:18.087680 13531 webserver.cc:460] Webserver started at http://127.21.27.190:42109/ using document root <none> and password file <none>
I20260809 22:49:18.088291 13531 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:18.088428 13531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:18.088691 13531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:18.091609 13531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "326b7dfcd77341809b8129bb9f1341b4"
format_stamp: "Formatted at 2026-08-09 22:49:18 on dist-test-slave-chsk"
server_key: "aa7b6304bd3f1c443a982d63ce969637"
server_key_iv: "35aca4887ecb335aef0eeb4f82bb5d24"
server_key_version: "encryptionkey@0"
I20260809 22:49:18.092338 13531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "326b7dfcd77341809b8129bb9f1341b4"
format_stamp: "Formatted at 2026-08-09 22:49:18 on dist-test-slave-chsk"
server_key: "aa7b6304bd3f1c443a982d63ce969637"
server_key_iv: "35aca4887ecb335aef0eeb4f82bb5d24"
server_key_version: "encryptionkey@0"
I20260809 22:49:18.097294 13531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:18.100623 13546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:18.101858 13531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:49:18.101987 13531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "326b7dfcd77341809b8129bb9f1341b4"
format_stamp: "Formatted at 2026-08-09 22:49:18 on dist-test-slave-chsk"
server_key: "aa7b6304bd3f1c443a982d63ce969637"
server_key_iv: "35aca4887ecb335aef0eeb4f82bb5d24"
server_key_version: "encryptionkey@0"
I20260809 22:49:18.102164 13531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:18.120729 13531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:18.124799 13531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:18.125036 13531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:18.144085 13531 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:42051
I20260809 22:49:18.144125 13601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:42051 every 8 connection(s)
I20260809 22:49:18.145443 13531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:18.149179 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:18.154361 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13531
I20260809 22:49:18.154557 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:18.155072 21614 external_mini_cluster.cc:1428] Setting key 8051492e9715366e10b20749e4bcbc1d
I20260809 22:49:18.160537 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4: Bootstrap starting.
I20260809 22:49:18.164088 13603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:18.165354 13603 log.cc:824] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:18.168591 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4: No bootstrap required, opened a new log
I20260809 22:49:18.168766 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:18.156896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47745 (local address 127.21.27.190:42051)
0809 22:49:18.157766 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:18.157776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:18.157815 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:49:18.159614 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:18.159636 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:18.160004 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:18.160321 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:18.160953 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:18.162262 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:18.164161 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:18.164865 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:18.165025 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:18.166339 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:18.166350 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:18.166369 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:18.167044 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:18.167117 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:18.167187 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:18.167280 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:18.167504 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:49:18.167748 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":435,"threads_started":1}
I20260809 22:49:18.172358 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:18.178435 13603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } }
I20260809 22:49:18.178934 13603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:18.179054 13603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326b7dfcd77341809b8129bb9f1341b4, State: Initialized, Role: FOLLOWER
I20260809 22:49:18.179747 13603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } }
I20260809 22:49:18.180043 13603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:18.180193 13603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:18.180361 13603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:18.183661 13603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } }
I20260809 22:49:18.184264 13603 leader_election.cc:302] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 326b7dfcd77341809b8129bb9f1341b4; no voters:
I20260809 22:49:18.185081 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:18.177669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59239 (local address 127.21.27.190:42051)
0809 22:49:18.177848 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:18.177854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:18.178345 (+ 491us) server_negotiation.cc:407] Connection header received
0809 22:49:18.178500 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:18.178506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:18.178584 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:18.178696 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:18.179344 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:18.180459 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:18.181741 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:18.182944 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:18.183072 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:18.184174 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:18.184179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:18.184184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:18.184204 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:18.184256 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:18.184280 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:18.184380 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:18.184536 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 22:49:18.184628 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:49:18.185071 13603 leader_election.cc:288] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:18.185139 13613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:18.186924 13613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [term 1 LEADER]: Becoming Leader. State: Replica: 326b7dfcd77341809b8129bb9f1341b4, State: Running, Role: LEADER
I20260809 22:49:18.187486 13613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } }
I20260809 22:49:18.187629 13603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:18.190333 13615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 326b7dfcd77341809b8129bb9f1341b4. Latest consensus state: current_term: 1 leader_uuid: "326b7dfcd77341809b8129bb9f1341b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } } }
I20260809 22:49:18.190933 13615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:18.198153 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "326b7dfcd77341809b8129bb9f1341b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b7dfcd77341809b8129bb9f1341b4" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 42051 } } }
I20260809 22:49:18.198415 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:18.198683 13620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:18.203303 13620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:18.211949 13620 catalog_manager.cc:1347] Generated new cluster ID: 37e8c56630564ad5bbb5bb3e1c1e4099
I20260809 22:49:18.212061 13620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:18.228209 13620 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:18.229444 13620 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:18.236502 13620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 326b7dfcd77341809b8129bb9f1341b4: Generated new TSK 0
I20260809 22:49:18.237424 13620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:18.253412 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13531
W20260809 22:49:18.254509 13597 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47745
W20260809 22:49:18.255227 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:18.255367 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13531
2026-08-09T22:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:18 dist-test-slave-chsk krb5kdc[13636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:18 dist-test-slave-chsk krb5kdc[13636](info): set up 2 sockets
Aug 09 22:49:18 dist-test-slave-chsk krb5kdc[13636](info): commencing operation
krb5kdc: starting...
W20260809 22:49:19.420258 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:19 dist-test-slave-chsk krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:19.754614 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35041
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:45235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:35041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:20.094463 13649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:20.094811 13649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:20.094990 13649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:20.102404 13649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:20.102561 13649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:20.102672 13649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:20.102789 13649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:20.102880 13649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:20.112635 13649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:45235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:35041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:35041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:20.113972 13649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:20.115301 13649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:20.127518 13658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:20.127497 13660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:20.127604 13656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:20.128233 13649 server_base.cc:1034] running on GCE node
I20260809 22:49:20.128989 13649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:20.130326 13649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:20.131557 13649 hybrid_clock.cc:648] HybridClock initialized: now 1786315760131526 us; error 50 us; skew 500 ppm
I20260809 22:49:20.131904 13649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:20 dist-test-slave-chsk krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:20.138626 13649 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:20.140215 13649 webserver.cc:460] Webserver started at http://127.21.27.190:34617/ using document root <none> and password file <none>
I20260809 22:49:20.140905 13649 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:20.141054 13649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:20.141371 13649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:20.144438 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "ad4d21f3417748d2995cbae4c9daa741"
format_stamp: "Formatted at 2026-08-09 22:49:20 on dist-test-slave-chsk"
server_key: "27cd78a15b52a8b2a5cef05704be8250"
server_key_iv: "bac7412fa1a33c517fdedb02d3e8a980"
server_key_version: "encryptionkey@0"
I20260809 22:49:20.145210 13649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "ad4d21f3417748d2995cbae4c9daa741"
format_stamp: "Formatted at 2026-08-09 22:49:20 on dist-test-slave-chsk"
server_key: "27cd78a15b52a8b2a5cef05704be8250"
server_key_iv: "bac7412fa1a33c517fdedb02d3e8a980"
server_key_version: "encryptionkey@0"
I20260809 22:49:20.150264 13649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:49:20.153621 13667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:20.154829 13649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:20.155021 13649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "ad4d21f3417748d2995cbae4c9daa741"
format_stamp: "Formatted at 2026-08-09 22:49:20 on dist-test-slave-chsk"
server_key: "27cd78a15b52a8b2a5cef05704be8250"
server_key_iv: "bac7412fa1a33c517fdedb02d3e8a980"
server_key_version: "encryptionkey@0"
I20260809 22:49:20.155309 13649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:20.161047 13649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:20.165038 13649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:20.165354 13649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:20.184062 13649 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:35041
I20260809 22:49:20.184095 13723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:35041 every 8 connection(s)
I20260809 22:49:20.185468 13649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:20.186960 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13649
I20260809 22:49:20.187175 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:20.187783 21614 external_mini_cluster.cc:1428] Setting key 0de7528b717882988fe4da7d2e94a87a
I20260809 22:49:20.190083 13724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:20.195514 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:20.189624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33915 (local address 127.21.27.190:35041)
0809 22:49:20.190467 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:20.190479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:20.190519 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:49:20.192146 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:20.192165 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:20.192486 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:20.192797 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:20.193884 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:20.193890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:20.193901 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:20.193918 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:20.194001 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:20.194020 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:20.194209 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:20.194353 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 22:49:20.194553 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":375,"threads_started":1}
I20260809 22:49:20.199123 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:20.202172 13724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741: Bootstrap starting.
I20260809 22:49:20.205945 13724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:20.206197 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:20.203680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44189 (local address 127.21.27.190:35041)
0809 22:49:20.203881 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:20.203887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:20.204228 (+ 341us) server_negotiation.cc:407] Connection header received
0809 22:49:20.204426 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:20.204429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:20.204474 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:20.204549 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:20.205503 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:20.205507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:20.205510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:49:20.205523 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:20.205558 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:49:20.205572 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:20.205660 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:20.205822 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 22:49:20.205864 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:49:20.207228 13724 log.cc:824] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:20.209810 13724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741: No bootstrap required, opened a new log
I20260809 22:49:20.216848 13724 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } }
I20260809 22:49:20.217217 13724 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:49:20.217325 13724 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4d21f3417748d2995cbae4c9daa741, State: Initialized, Role: FOLLOWER
I20260809 22:49:20.217929 13724 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } }
I20260809 22:49:20.218163 13724 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:49:20.218314 13724 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:49:20.218508 13724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:20.220795 13724 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } }
I20260809 22:49:20.221277 13724 leader_election.cc:302] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad4d21f3417748d2995cbae4c9daa741; no voters:
I20260809 22:49:20.221849 13724 leader_election.cc:288] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:49:20.222056 13734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:20.222704 13734 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [term 1 LEADER]: Becoming Leader. State: Replica: ad4d21f3417748d2995cbae4c9daa741, State: Running, Role: LEADER
I20260809 22:49:20.223323 13734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } }
I20260809 22:49:20.224074 13724 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:20.226718 13736 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad4d21f3417748d2995cbae4c9daa741. Latest consensus state: current_term: 1 leader_uuid: "ad4d21f3417748d2995cbae4c9daa741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } } }
I20260809 22:49:20.226717 13735 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad4d21f3417748d2995cbae4c9daa741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4d21f3417748d2995cbae4c9daa741" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 35041 } } }
I20260809 22:49:20.227319 13736 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:20.227332 13735 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:20.231774 13742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:20.236410 13742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:20.245081 13742 catalog_manager.cc:1347] Generated new cluster ID: 008c8bc2776c44e8af11008343b24a35
I20260809 22:49:20.245204 13742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:20.258306 13742 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:20.258796 13742 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:20.269141 13742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad4d21f3417748d2995cbae4c9daa741: Generated new TSK 0
I20260809 22:49:20.269917 13742 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:20.283094 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13649
W20260809 22:49:20.284735 13719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33915
W20260809 22:49:20.285395 21614 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:49:20.285535 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13649
2026-08-09T22:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2030 ms)
[----------] 18 tests from AuthTokenIssuingTest (37541 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:20 dist-test-slave-chsk krb5kdc[13756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:20 dist-test-slave-chsk krb5kdc[13756](info): set up 2 sockets
Aug 09 22:49:20 dist-test-slave-chsk krb5kdc[13756](info): commencing operation
krb5kdc: starting...
W20260809 22:49:21.468317 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:21 dist-test-slave-chsk krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:21.786288 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40149
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:22.122116 13771 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:22.122490 13771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:22.122613 13771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:22.122764 13771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:22.130123 13771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:22.130244 13771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:22.130350 13771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:22.130434 13771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:22.130525 13771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:22.140259 13771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:40149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:22.141597 13771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:22.142886 13771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:22.154585 13776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.154620 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.154747 13780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:22.155835 13771 server_base.cc:1034] running on GCE node
I20260809 22:49:22.156564 13771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:22.157851 13771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:22.159096 13771 hybrid_clock.cc:648] HybridClock initialized: now 1786315762159039 us; error 33 us; skew 500 ppm
I20260809 22:49:22.159448 13771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:22.162520 13771 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:22.163959 13771 webserver.cc:460] Webserver started at http://127.21.27.190:36811/ using document root <none> and password file <none>
I20260809 22:49:22.164604 13771 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:22.164734 13771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:22.165052 13771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:22.167927 13771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "7240d1a82d7e7e0169f355a8b4234b14"
server_key_iv: "5b394577dcacf16ecb7af544958bee0c"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.168699 13771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "7240d1a82d7e7e0169f355a8b4234b14"
server_key_iv: "5b394577dcacf16ecb7af544958bee0c"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.173629 13771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:49:22.176934 13789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.178190 13771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:49:22.178361 13771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "7240d1a82d7e7e0169f355a8b4234b14"
server_key_iv: "5b394577dcacf16ecb7af544958bee0c"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.178570 13771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:22.217123 13771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:22.221195 13771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:22.221495 13771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:22.240451 13771 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:40149
I20260809 22:49:22.240476 13844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:40149 every 8 connection(s)
I20260809 22:49:22.241770 13771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:22.245266 13845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:22.245802 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13771
I20260809 22:49:22.246023 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:22.246688 21614 external_mini_cluster.cc:1428] Setting key 586afb820754542b43d97f829e09613e
I20260809 22:49:22.251808 13845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:22.274145 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.254894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.21.27.190:40149)
0809 22:49:22.255843 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.255859 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.256276 (+ 417us) server_negotiation.cc:407] Connection header received
0809 22:49:22.258168 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.258173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.258377 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.258584 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.260248 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.261852 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.263272 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.263817 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.264020 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.267284 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.267292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.267302 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.267333 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.269953 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.270621 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.270627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.270629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.270779 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.271190 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.271199 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.271201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.271633 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.271839 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.272404 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.272693 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":458,"threads_started":1}
I20260809 22:49:22.274168 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.254908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40149 (local address 127.0.0.1:44052)
0809 22:49:22.256127 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.256160 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:22.257829 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:22.258593 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:22.258599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:22.259478 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:22.260001 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.260024 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.262059 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.262064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.263089 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.263101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.264053 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.264060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.264246 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.264912 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:22.264943 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:22.267046 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:22.270095 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.270106 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.270116 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.270482 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.270892 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.270896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.270899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.271056 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.271762 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:22.271774 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:22.271984 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 22:49:22.272228 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 22:49:22.272840 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":152,"thread_start_us":815,"threads_started":1}
WARNING: no policy specified for kudu/127.21.27.189@KRBTEST.COM; defaulting to no policy
I20260809 22:49:22.276940 13845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:22.278059 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.277681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50184)
0809 22:49:22.277839 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.277946 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 22:49:22.278337 13791 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.189@KRBTEST.COM" created.
I20260809 22:49:22.279814 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.279461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50198)
0809 22:49:22.279602 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.279673 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:22.280081 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:34659: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
I20260809 22:49:22.313967 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:34659
--webserver_interface=127.21.27.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:22.329406 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } attempt: 1
I20260809 22:49:22.330541 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.330074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50214)
0809 22:49:22.330274 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.330411 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:49:22.331641 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.331286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50218)
0809 22:49:22.331432 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.331493 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:49:22.331863 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:34659: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
I20260809 22:49:22.400115 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } attempt: 2
I20260809 22:49:22.401453 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.400944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50222)
0809 22:49:22.401215 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.401309 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 22:49:22.402451 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.402124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50236)
0809 22:49:22.402248 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.402324 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:49:22.402675 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:34659: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
I20260809 22:49:22.493937 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } attempt: 3
I20260809 22:49:22.495225 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.494744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50238)
0809 22:49:22.494936 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.495042 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 22:49:22.496302 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.495992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50244)
0809 22:49:22.496112 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.496175 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:49:22.496484 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:34659: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
I20260809 22:49:22.639753 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } attempt: 4
I20260809 22:49:22.640973 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.640402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50260)
0809 22:49:22.640724 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.640839 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260809 22:49:22.641933 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.641607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50274)
0809 22:49:22.641746 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.641824 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:49:22.642143 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:34659: Network error: Client connection negotiation failed: client connection to 127.21.27.189:34659: connect: Connection refused (error 111)
W20260809 22:49:22.654630 13853 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:22.654983 13853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:22.655124 13853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:22.655275 13853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:22.662945 13853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:22.663057 13853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:22.663192 13853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:22.663285 13853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:22.663378 13853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:22.673058 13853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.27.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:22.674353 13853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:22.675674 13853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:22.687423 13858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.687491 13862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:22.687458 13859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:22.688753 13853 server_base.cc:1034] running on GCE node
I20260809 22:49:22.689493 13853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:22.690773 13853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:22.692001 13853 hybrid_clock.cc:648] HybridClock initialized: now 1786315762691931 us; error 46 us; skew 500 ppm
I20260809 22:49:22.692309 13853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:22.695508 13853 init.cc:377] Logged in from keytab as kudu/127.21.27.189@KRBTEST.COM (short username kudu)
I20260809 22:49:22.696871 13853 webserver.cc:460] Webserver started at http://127.21.27.189:36819/ using document root <none> and password file <none>
I20260809 22:49:22.697530 13853 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:22.697656 13853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:22.697973 13853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:22.700847 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data/instance:
uuid: "c1bdd4036a664604a7782b0a012397cb"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "41e85f0a0be0e02cc83d250ade3c5e9b"
server_key_iv: "cc5b418e9642a1a48244df2909ddfa89"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.701587 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal/instance:
uuid: "c1bdd4036a664604a7782b0a012397cb"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "41e85f0a0be0e02cc83d250ade3c5e9b"
server_key_iv: "cc5b418e9642a1a48244df2909ddfa89"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.706542 13853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 22:49:22.709736 13870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:22.710893 13853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 22:49:22.711066 13853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "c1bdd4036a664604a7782b0a012397cb"
format_stamp: "Formatted at 2026-08-09 22:49:22 on dist-test-slave-chsk"
server_key: "41e85f0a0be0e02cc83d250ade3c5e9b"
server_key_iv: "cc5b418e9642a1a48244df2909ddfa89"
server_key_version: "encryptionkey@0"
I20260809 22:49:22.711316 13853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:22.732170 13853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:22.735947 13853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:22.736215 13853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:22.754127 13853 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.189:34659
I20260809 22:49:22.754148 13924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.189:34659 every 8 connection(s)
I20260809 22:49:22.755472 13853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
I20260809 22:49:22.758961 13925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:22.763958 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13853
I20260809 22:49:22.764147 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-1/wal/instance
I20260809 22:49:22.764561 21614 external_mini_cluster.cc:1428] Setting key 6bc2752021caca06e2170f20f41674b1
I20260809 22:49:22.765342 13925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:22.787285 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.767836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44068 (local address 127.21.27.190:40149)
0809 22:49:22.768052 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.768058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.769175 (+ 1117us) server_negotiation.cc:407] Connection header received
0809 22:49:22.771965 (+ 2790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.771969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.772037 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.772126 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.774125 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.774786 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.777109 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.777539 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.777634 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.781079 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.781088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.781096 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.781143 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.784319 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.785058 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.785063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.785065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.785184 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.785583 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.785588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.785591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.785873 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.786026 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.786692 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.786919 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:49:22.788146 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.768251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40149 (local address 127.0.0.1:44068)
0809 22:49:22.769002 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.769049 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:22.771701 (+ 2652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:22.772382 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:22.772394 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:22.773160 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:22.773958 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.773986 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.774941 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.774949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.776965 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.776983 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.777764 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.777773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.777992 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.778677 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:22.778700 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:22.780849 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:22.784500 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.784506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.784517 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.784930 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.785298 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.785303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.785306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.785469 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.786008 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:22.786016 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:22.786313 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 22:49:22.786562 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 22:49:22.787075 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":275,"threads_started":1}
I20260809 22:49:22.789860 13925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.27.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.188@KRBTEST.COM" created.
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:22.806471 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.790956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.21.27.189:34659)
0809 22:49:22.791599 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.791605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.791629 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:22.791671 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.791676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.791831 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.792008 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.792953 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.794011 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.795824 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.796387 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.796478 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.800774 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.800784 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.800797 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.800825 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.803600 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.804406 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.804413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.804415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.804532 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.804930 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.804933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.804935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.805242 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.805436 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.805859 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.806180 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":510,"thread_start_us":385,"threads_started":1}
I20260809 22:49:22.806558 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.790760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50276)
0809 22:49:22.790916 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.790945 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:22.791022 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:22.792036 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:22.792044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:22.792550 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:22.792779 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.792792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.794156 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.794161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.795678 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.795696 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.796538 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.796546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.796707 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.797210 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:22.797234 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:22.800560 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:22.803723 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.803731 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.803737 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.804294 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.804627 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.804634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.804640 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.804828 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.805394 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:22.805405 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:22.805583 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 22:49:22.805739 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:49:22.806066 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 22:49:22.809121 13925 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:22.810165 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.809771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58572)
0809 22:49:22.809915 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.810025 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 22:49:22.810391 13872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:22.811975 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.811643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58580)
0809 22:49:22.811812 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.811880 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 22:49:22.812225 13925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
I20260809 22:49:22.834520 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:34685
--webserver_interface=127.21.27.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:22.861513 13925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 1
I20260809 22:49:22.862848 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.862203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58586)
0809 22:49:22.862463 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.862611 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:49:22.864063 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.863576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58600)
0809 22:49:22.863794 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.863899 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:49:22.864284 13925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:22.932535 13925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 2
I20260809 22:49:22.933847 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.933282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58612)
0809 22:49:22.933508 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.933637 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 22:49:22.934883 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.934504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58620)
0809 22:49:22.934646 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.934729 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:22.935086 13925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:22.941375 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } attempt: 5
Aug 09 22:49:22 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:22.954850 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.942176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50290 (local address 127.21.27.189:34659)
0809 22:49:22.942350 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:22.942356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:22.942452 (+ 96us) server_negotiation.cc:407] Connection header received
0809 22:49:22.942651 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:22.942661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:22.942744 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:22.942849 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:22.943713 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.944416 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.945558 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.946003 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.946134 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.949298 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:22.949303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:22.949306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:22.949336 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:22.952322 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.952945 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.952952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.952956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.953082 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:22.953434 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:22.953440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:22.953443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:22.953749 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:22.953913 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:22.954294 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 22:49:22.954525 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:49:22.954887 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.942141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50290)
0809 22:49:22.942349 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.942371 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:22.942528 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:22.942884 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:22.942889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:22.943371 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:22.943564 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.943573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.944545 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.944552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.945409 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:22.945421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:22.946146 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:22.946151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:22.946413 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:22.947045 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:22.947064 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:22.949110 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:22.952458 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.952464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.952468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.952833 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.953189 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:22.953194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:22.953197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:22.953330 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:22.953878 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:22.953883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:22.954034 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 22:49:22.954172 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 22:49:22.954507 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 22:49:22.956070 13845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:22.957031 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.956693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58624)
0809 22:49:22.956832 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.956902 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:49:22.957898 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:22.957566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58640)
0809 22:49:22.957711 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:22.957794 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 22:49:22.958107 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:23.009364 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 1
I20260809 22:49:23.010818 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.010092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58646)
0809 22:49:23.010382 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.010546 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 22:49:23.011922 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.011587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58658)
0809 22:49:23.011745 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.011816 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 22:49:23.012173 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:23.026340 13925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 3
I20260809 22:49:23.027585 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.026983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58666)
0809 22:49:23.027254 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.027376 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 22:49:23.028857 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.028285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58674)
0809 22:49:23.028560 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.028677 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260809 22:49:23.029153 13925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:23.080411 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 2
I20260809 22:49:23.081669 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.081138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58684)
0809 22:49:23.081387 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.081477 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:49:23.082924 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.082487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58688)
0809 22:49:23.082651 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.082753 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 22:49:23.083235 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
W20260809 22:49:23.171888 13934 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:23.172232 13934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:23.172366 13934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:23.172520 13934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 22:49:23.172400 13925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 4
I20260809 22:49:23.173617 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.173046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58694)
0809 22:49:23.173303 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.173412 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 22:49:23.174628 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.174257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58706)
0809 22:49:23.174408 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.174492 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:23.174850 13925 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
W20260809 22:49:23.180467 13934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:23.180546 13934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:23.180658 13934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:23.180740 13934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:23.180827 13934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:23.189471 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 3
I20260809 22:49:23.190587 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.190130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58720)
0809 22:49:23.190327 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.190415 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 22:49:23.191349 13934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:40149,127.21.27.189:34659,127.21.27.188:34685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:34685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.27.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:23.191710 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.191304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58734)
0809 22:49:23.191486 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.191583 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:49:23.191946 13845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:34685: Network error: Client connection negotiation failed: client connection to 127.21.27.188:34685: connect: Connection refused (error 111)
I20260809 22:49:23.192822 13934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:23.194118 13934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:23.205698 13944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.205670 13941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:23.205874 13942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:23.206493 13934 server_base.cc:1034] running on GCE node
I20260809 22:49:23.207286 13934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:23.208565 13934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:23.209776 13934 hybrid_clock.cc:648] HybridClock initialized: now 1786315763209726 us; error 29 us; skew 500 ppm
I20260809 22:49:23.210105 13934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:23.213232 13934 init.cc:377] Logged in from keytab as kudu/127.21.27.188@KRBTEST.COM (short username kudu)
I20260809 22:49:23.214740 13934 webserver.cc:460] Webserver started at http://127.21.27.188:45571/ using document root <none> and password file <none>
I20260809 22:49:23.215529 13934 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:23.215656 13934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:23.215998 13934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:23.218894 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data/instance:
uuid: "648f15c5c5ae40fdb13c607302f81ded"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-chsk"
server_key: "02e95ba53cd49fff7f3614e5faa3c000"
server_key_iv: "fe5e42638107c536b6f57dbe352eb02a"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.219712 13934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal/instance:
uuid: "648f15c5c5ae40fdb13c607302f81ded"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-chsk"
server_key: "02e95ba53cd49fff7f3614e5faa3c000"
server_key_iv: "fe5e42638107c536b6f57dbe352eb02a"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.224619 13934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:49:23.227787 13952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:23.229044 13934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:49:23.229223 13934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "648f15c5c5ae40fdb13c607302f81ded"
format_stamp: "Formatted at 2026-08-09 22:49:23 on dist-test-slave-chsk"
server_key: "02e95ba53cd49fff7f3614e5faa3c000"
server_key_iv: "fe5e42638107c536b6f57dbe352eb02a"
server_key_version: "encryptionkey@0"
I20260809 22:49:23.229439 13934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:23.255053 13934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:23.259446 13934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:23.259734 13934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:23.277796 13934 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.188:34685
I20260809 22:49:23.277804 14009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.188:34685 every 8 connection(s)
I20260809 22:49:23.279153 13934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
I20260809 22:49:23.282744 14010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:23.284518 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 13934
I20260809 22:49:23.284695 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786315394803050-21614-0/minicluster-data/master-2/wal/instance
I20260809 22:49:23.285137 21614 external_mini_cluster.cc:1428] Setting key 28c3718f16feb5d5551c3ecfd089ea2a
I20260809 22:49:23.292177 14010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:23.309499 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.287028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.21.27.188:34685)
0809 22:49:23.287925 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.287940 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.287997 (+ 57us) server_negotiation.cc:407] Connection header received
0809 22:49:23.289759 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.289787 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.290303 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.290684 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.292175 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.294133 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.295675 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.296496 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.296678 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.301551 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.301562 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.301577 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.301620 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.305038 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.305725 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.305733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.305741 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.305934 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.306358 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.306363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.306366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.307054 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.307424 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.307808 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.308225 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":346,"threads_started":1}
I20260809 22:49:23.311555 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.295109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:40149 (local address 127.0.0.1:44082)
0809 22:49:23.296080 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.296129 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.296286 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.296688 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.296695 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.297554 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.297905 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.297929 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.298990 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.298998 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.300311 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.300323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.301390 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.301395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.301602 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.302307 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.302324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.305064 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.308853 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.308862 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.308867 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.309276 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.309733 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.309738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.309741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.309912 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.310435 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.310443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.310599 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.310785 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.311102 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":589,"threads_started":1}
I20260809 22:49:23.311709 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.295119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.21.27.190:40149)
0809 22:49:23.295695 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.295701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.296223 (+ 522us) server_negotiation.cc:407] Connection header received
0809 22:49:23.296413 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.296419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.296500 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.296613 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.298085 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.298835 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.300476 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.301193 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.301327 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.305266 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.305271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.305274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.305306 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.308734 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.309406 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.309411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.309416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.309581 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.310029 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.310034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.310037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.310299 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.310463 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.310901 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.311116 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":256,"threads_started":1}
I20260809 22:49:23.312990 14010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:23.328557 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.313918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.21.27.189:34659)
0809 22:49:23.314113 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.314119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.314194 (+ 75us) server_negotiation.cc:407] Connection header received
0809 22:49:23.314378 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.314384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.314459 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.314567 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.315441 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.316494 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.318190 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.318843 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.318971 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.322696 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.322703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.322707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.322740 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.325767 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.326530 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.326534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.326536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.326650 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.327052 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.327059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.327064 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.327478 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.327642 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.328078 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.328299 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:49:23.328560 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.313760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:34659 (local address 127.0.0.1:50292)
0809 22:49:23.314121 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.314141 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.314253 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.314600 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.314604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.315081 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.315301 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.315312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.316628 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.316633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.318030 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.318044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.319014 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.319020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.319252 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.319952 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.319976 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.322509 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.325898 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.325906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.325911 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.326421 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.326756 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.326761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.326764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.326944 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.327607 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.327615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.327800 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.327955 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.328203 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252}
I20260809 22:49:23.329751 14010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:23.330045 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.313185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.21.27.190:40149)
0809 22:49:23.313353 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.313359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.313380 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:49:23.313426 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.313431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.313501 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.313608 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.315294 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.316390 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.318190 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.318843 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.318974 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.323835 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.323840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.323843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.323874 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.326871 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.327638 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.327653 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.327656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.327800 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.328241 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.328246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.328249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.328597 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.328893 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.329510 (+ 617us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.329726 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315763, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:23.343537 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.330536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.21.27.188:34685)
0809 22:49:23.330725 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.330730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.330803 (+ 73us) server_negotiation.cc:407] Connection header received
0809 22:49:23.330957 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.330960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.331012 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.331113 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.331805 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.332803 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.334130 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.334549 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.334700 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.338308 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.338313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.338316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.338343 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.340904 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.341650 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.341657 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.341661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.341814 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.342222 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.342225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.342227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.342491 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.342632 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.343052 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.343294 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:49:23.343894 14015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.330484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58756)
0809 22:49:23.330716 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.330736 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.330859 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.331126 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.331130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.331529 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.331684 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.331699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.332936 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.332943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.334004 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.334018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.334687 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.334691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.334878 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.335492 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.335508 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.338129 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.341027 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.341033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.341037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.341537 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.341928 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.341933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.341937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.342111 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.342608 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.342613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.342785 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.342966 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.343503 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":133}
I20260809 22:49:23.347067 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.331838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50304 (local address 127.21.27.189:34659)
0809 22:49:23.332005 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.332010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.332030 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:49:23.332161 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.332167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.332247 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.332337 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.333587 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.334475 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.335796 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.336449 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.336592 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.340908 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.340913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.340917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.340947 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.344539 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.345314 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.345318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.345320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.345442 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.345865 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.345868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.345870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.346125 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.346338 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.346672 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.346844 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:49:23.349268 14010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: Bootstrap starting.
I20260809 22:49:23.351716 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:23.352648 14010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.353682 14010 log.cc:824] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.356881 14010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: No bootstrap required, opened a new log
I20260809 22:49:23.354504 13817 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44090
I20260809 22:49:23.367805 14010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.368233 14010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648f15c5c5ae40fdb13c607302f81ded, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.368754 14010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.369148 13845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 4
I20260809 22:49:23.369926 14017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.370343 14017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.371416 14010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
W20260809 22:49:23.383714 14029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:49:23.383867 14029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:49:23.386871 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.369906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58770)
0809 22:49:23.370190 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.370212 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.370353 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.370757 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.370762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.371258 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.371416 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.371425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.372475 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.372483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.374000 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.374017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.375134 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.375161 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.375410 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.376121 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.376146 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.379207 (+ 3061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.383721 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.383727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.383734 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.384281 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.384764 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.384769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.384772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.384939 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.385795 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.385801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.385992 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.386147 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.386453 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260809 22:49:23.387600 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.369998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.21.27.188:34685)
0809 22:49:23.370192 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.370200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.370304 (+ 104us) server_negotiation.cc:407] Connection header received
0809 22:49:23.370525 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.370530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.370606 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.370723 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.371572 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.372326 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.374208 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.374935 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.375091 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.379407 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.379417 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.379424 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.379465 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.383542 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.384430 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.384440 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.384443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.384608 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.385085 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.385095 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.385098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.385564 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.385690 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.386407 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.386856 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 22:49:23.379882 13934 master_runner.cc:186] Error getting master registration for 127.21.27.190:40149: OK, instance_id { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" instance_seqno: 1786315762235270 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:23.392701 13934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:23.393515 13845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: Bootstrap starting.
I20260809 22:49:23.397140 13845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.398435 13845 log.cc:824] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.401706 13845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: No bootstrap required, opened a new log
I20260809 22:49:23.409405 13845 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.409843 13845 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde2f90cfbbf4bdaaa2c9f77e4eb133f, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.410342 13845 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.411435 14032 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.411865 14032 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.412632 13845 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:23.412669 13817 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:49:23.412933 13817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:23.417039 13817 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.420265 13817 leader_election.cc:288] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [CANDIDATE]: Term 1 election: Requested vote from peers c1bdd4036a664604a7782b0a012397cb (127.21.27.189:34659), 648f15c5c5ae40fdb13c607302f81ded (127.21.27.188:34685)
I20260809 22:49:23.421097 13983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "648f15c5c5ae40fdb13c607302f81ded"
I20260809 22:49:23.421607 13983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:23.425011 13983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fde2f90cfbbf4bdaaa2c9f77e4eb133f in term 1.
I20260809 22:49:23.425623 13791 leader_election.cc:302] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648f15c5c5ae40fdb13c607302f81ded, fde2f90cfbbf4bdaaa2c9f77e4eb133f; no voters:
I20260809 22:49:23.426142 14032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 1 FOLLOWER]: Leader election won for term 1
W20260809 22:49:23.435578 14044 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:49:23.435870 14044 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:49:23.438838 14032 raft_consensus.cc:686] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [term 1 LEADER]: Becoming Leader. State: Replica: fde2f90cfbbf4bdaaa2c9f77e4eb133f, State: Running, Role: LEADER
I20260809 22:49:23.439550 14032 consensus_queue.cc:237] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.447956 14048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde2f90cfbbf4bdaaa2c9f77e4eb133f. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.448689 14048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:23.424414 13771 master_runner.cc:186] Error getting master registration for 127.21.27.189:34659: OK, instance_id { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" instance_seqno: 1786315762749086 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:23.451181 13771 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:23.457147 14053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:23.460321 14053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:23.474007 13925 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } attempt: 5
W20260809 22:49:23.474421 13795 tablet.cc:2369] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:49:23 dist-test-slave-chsk krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315762, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:49:23.497066 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.474644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:34685 (local address 127.0.0.1:58774)
0809 22:49:23.475230 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:23.475254 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:23.475389 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:23.476166 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:23.476175 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:23.476861 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:23.477162 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.477178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.479296 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.479305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.481013 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:23.481031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.482531 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.482540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:23.482875 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.484167 (+ 1292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:23.484199 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:23.487735 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:23.493564 (+ 5829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.493572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.493579 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.494275 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.494814 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:23.494821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:23.494827 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:23.495020 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:23.495876 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:23.495884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:23.496094 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 22:49:23.496285 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 22:49:23.496658 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441}
I20260809 22:49:23.499083 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.475174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.21.27.188:34685)
0809 22:49:23.475496 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.475505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.475534 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:49:23.475619 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.475625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.475736 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.475886 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.477415 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.478771 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.481216 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.482081 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.482243 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.488028 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.488039 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.488047 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.488102 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.493312 (+ 5210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.494451 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.494460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.494466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.494656 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.495202 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.495210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.495216 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.495628 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.495802 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.498437 (+ 2635us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.498701 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260809 22:49:23.510802 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: Bootstrap starting.
W20260809 22:49:23.511704 13959 tablet.cc:2369] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:23.517678 13983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [term 1 FOLLOWER]: Refusing update from remote peer fde2f90cfbbf4bdaaa2c9f77e4eb133f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:23.518882 14048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [LEADER]: Connected to new peer: Peer: permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:23.526357 13925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:23.528204 13925 log.cc:824] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:23.561970 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.472511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.21.27.189:34659)
0809 22:49:23.472705 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.472711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.484931 (+ 12220us) server_negotiation.cc:407] Connection header received
0809 22:49:23.501651 (+ 16720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.501659 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.501740 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.501811 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.510240 (+ 8429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.511368 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.516207 (+ 4839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.516859 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.516972 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.531759 (+ 14787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.531770 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.531778 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.531834 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.536556 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.546654 (+ 10098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.546664 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.546671 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.547002 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.549221 (+ 2219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.549229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.549258 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.550063 (+ 805us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.550574 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.561098 (+ 10524us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.561452 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:49:23.573815 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.480653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.21.27.190:40149)
0809 22:49:23.480839 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.480845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.497121 (+ 16276us) server_negotiation.cc:407] Connection header received
0809 22:49:23.499159 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.499166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.499259 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.499351 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.508859 (+ 9508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.510012 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.520560 (+ 10548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.521232 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.521335 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.541022 (+ 19687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.541034 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.541041 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.541119 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.545871 (+ 4752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.560644 (+ 14773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.560654 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.560660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.561039 (+ 379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.566272 (+ 5233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.566278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.566281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.566741 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.567054 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.572898 (+ 5844us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.573226 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:49:23.580128 14017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde2f90cfbbf4bdaaa2c9f77e4eb133f. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.580483 14017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.583089 14055 mvcc.cc:205] Tried to move back new op lower bound from 7316749367294033920 to 7316749367054585856. Current Snapshot: MvccSnapshot[applied={T|T < 7316749367294033920}]
I20260809 22:49:23.584574 14017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.584832 14017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.585498 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.485446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58782 (local address 127.21.27.188:34685)
0809 22:49:23.498570 (+ 13124us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.498576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.504661 (+ 6085us) server_negotiation.cc:407] Connection header received
0809 22:49:23.504727 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.504733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.504812 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.504938 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.507279 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.508647 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.518932 (+ 10285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.520176 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.520313 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.550956 (+ 30643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:23.550969 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:23.550977 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:23.551094 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:23.563415 (+ 12321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.567129 (+ 3714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.567135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.567385 (+ 250us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.570752 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:23.572212 (+ 1460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:23.572220 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:23.572226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:23.578375 (+ 6149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:23.582385 (+ 4010us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.582989 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.584330 (+ 1341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12970,"thread_start_us":13145,"threads_started":1}
I20260809 22:49:23.590070 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: No bootstrap required, opened a new log
I20260809 22:49:23.598519 14061 mvcc.cc:205] Tried to move back new op lower bound from 7316749367294033920 to 7316749367054585856. Current Snapshot: MvccSnapshot[applied={T|T < 7316749367294033920 or (T in {7316749367294033920})}]
I20260809 22:49:23.599766 14053 catalog_manager.cc:1347] Generated new cluster ID: 129ee579da704ae9b3f135058207a46d
I20260809 22:49:23.599880 14053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:23.601970 14048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.602278 14048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:23.604684 13925 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.605041 13925 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1bdd4036a664604a7782b0a012397cb, State: Initialized, Role: FOLLOWER
I20260809 22:49:23.605504 13925 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } }
I20260809 22:49:23.612418 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.612843 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.614715 13925 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:23.616564 13897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c1bdd4036a664604a7782b0a012397cb"
I20260809 22:49:23.617146 13897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:23.621860 13897 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fde2f90cfbbf4bdaaa2c9f77e4eb133f in term 1.
I20260809 22:49:23.624398 13896 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [term 1 FOLLOWER]: Refusing update from remote peer fde2f90cfbbf4bdaaa2c9f77e4eb133f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:23.625460 14032 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:23.645596 14053 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:23.651484 14053 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:23.678112 14053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f: Generated new TSK 0
I20260809 22:49:23.679216 14053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:23.682338 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde2f90cfbbf4bdaaa2c9f77e4eb133f. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.682770 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.683511 14079 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.683874 14079 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:23.684623 14048 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde2f90cfbbf4bdaaa2c9f77e4eb133f" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 40149 } } peers { permanent_uuid: "c1bdd4036a664604a7782b0a012397cb" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 34659 } } peers { permanent_uuid: "648f15c5c5ae40fdb13c607302f81ded" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 34685 } } }
I20260809 22:49:23.684921 14048 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde2f90cfbbf4bdaaa2c9f77e4eb133f [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:23.738715 14087 catalog_manager.cc:1259] Loaded cluster ID: 129ee579da704ae9b3f135058207a46d
I20260809 22:49:23.738831 14087 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: loading cluster ID for follower catalog manager: success
I20260809 22:49:23.747280 14087 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: acquiring CA information for follower catalog manager: success
I20260809 22:49:23.749902 14087 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c1bdd4036a664604a7782b0a012397cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 22:49:23.838933 21614 mini_kdc.cc:348] Time spent kdestroy: real 0.140s user 0.000s sys 0.033s
W20260809 22:49:23.888541 13929 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:23.879112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.21.27.189:34659)
0809 22:49:23.879349 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.879358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.885989 (+ 6631us) server_negotiation.cc:407] Connection header received
0809 22:49:23.886512 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.886522 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.886617 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.886693 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.888159 (+ 1466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50312: BlockingRecv error: recv got EOF from 127.0.0.1:50312 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260809 22:49:23.891448 14014 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:23.878724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44116 (local address 127.21.27.190:40149)
0809 22:49:23.878964 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.878974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.889688 (+ 10714us) server_negotiation.cc:407] Connection header received
0809 22:49:23.889965 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.889971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.890068 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.890160 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.891218 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44116: BlockingRecv error: recv got EOF from 127.0.0.1:44116 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 22:49:23.900246 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:23.897953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.21.27.188:34685)
0809 22:49:23.898149 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.898157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.898434 (+ 277us) server_negotiation.cc:407] Connection header received
0809 22:49:23.898709 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.898714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.898811 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.898902 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.899988 (+ 1086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58784: BlockingRecv error: recv got EOF from 127.0.0.1:58784 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:49:23.952356 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.928544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.21.27.189:34659)
0809 22:49:23.928794 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.928804 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.943773 (+ 14969us) server_negotiation.cc:407] Connection header received
0809 22:49:23.944068 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.944077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.944306 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.944491 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:23.945193 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.946390 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.948477 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.949160 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.949284 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.950056 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:23.950778 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:23.950907 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.951495 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.951594 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 22:49:23.983672 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:23.981160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.21.27.188:34685)
0809 22:49:23.981356 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.981362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.982038 (+ 676us) server_negotiation.cc:407] Connection header received
0809 22:49:23.982236 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.982243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.982372 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.982495 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:23.983415 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58800: BlockingRecv error: recv got EOF from 127.0.0.1:58800 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:49:23.995452 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:23.988039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.21.27.189:34659)
0809 22:49:23.988236 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:23.988242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:23.988709 (+ 467us) server_negotiation.cc:407] Connection header received
0809 22:49:23.988997 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:23.989003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:23.989110 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:23.989179 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:23.989881 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.991299 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.993290 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:23.994011 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:23.994148 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:23.994641 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:23.994790 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:23.994884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:23.995042 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 22:49:23.995120 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 22:49:24.009382 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.003188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.21.27.188:34685)
0809 22:49:24.003378 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.003384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.007727 (+ 4343us) server_negotiation.cc:407] Connection header received
0809 22:49:24.008019 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.008026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.008135 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.008246 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.009147 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58810: BlockingRecv error: recv got EOF from 127.0.0.1:58810 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:49:24.022125 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.014905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.21.27.189:34659)
0809 22:49:24.015127 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.015133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.015716 (+ 583us) server_negotiation.cc:407] Connection header received
0809 22:49:24.016055 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.016061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.016150 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.016239 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.016802 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.017979 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.019919 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.020587 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.020716 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.021241 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.021388 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.021477 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.021654 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.021737 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 22:49:24.031366 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.029596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.21.27.188:34685)
0809 22:49:24.029791 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.029797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.029817 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:49:24.030000 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.030005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.030095 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.030193 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.031079 (+ 886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58820: BlockingRecv error: recv got EOF from 127.0.0.1:58820 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:49:24.045804 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.038856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50344 (local address 127.21.27.189:34659)
0809 22:49:24.039104 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.039111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.039678 (+ 567us) server_negotiation.cc:407] Connection header received
0809 22:49:24.039852 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.039858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.039944 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.040066 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.040532 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.041728 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.043705 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.044402 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.044532 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.045039 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.045185 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.045280 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.045454 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.045535 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
W20260809 22:49:24.060946 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.058492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.21.27.188:34685)
0809 22:49:24.058747 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.058755 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.059349 (+ 594us) server_negotiation.cc:407] Connection header received
0809 22:49:24.059564 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.059570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.059679 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.059797 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.060711 (+ 914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58824: BlockingRecv error: recv got EOF from 127.0.0.1:58824 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
I20260809 22:49:24.080019 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.072788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.21.27.189:34659)
0809 22:49:24.072990 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.072997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.073534 (+ 537us) server_negotiation.cc:407] Connection header received
0809 22:49:24.073739 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.073747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.073844 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.073945 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.074482 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.075668 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.077682 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.078427 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.078561 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.079067 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.079303 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.079396 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.079581 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.079685 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 22:49:24.089315 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.086932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.21.27.188:34685)
0809 22:49:24.087133 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.087162 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.087701 (+ 539us) server_negotiation.cc:407] Connection header received
0809 22:49:24.087883 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.087889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.087985 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.088097 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.089079 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58830: BlockingRecv error: recv got EOF from 127.0.0.1:58830 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:49:24.117240 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.109571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.21.27.189:34659)
0809 22:49:24.109769 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.109781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.110393 (+ 612us) server_negotiation.cc:407] Connection header received
0809 22:49:24.110612 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.110618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.110713 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.110816 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.111356 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.112652 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.114831 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.115655 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.115831 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.116407 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.116538 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.116628 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.116803 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.116893 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 22:49:24.126802 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.124866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.21.27.188:34685)
0809 22:49:24.125038 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.125043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.125177 (+ 134us) server_negotiation.cc:407] Connection header received
0809 22:49:24.125489 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.125494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.125580 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.125679 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.126584 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58844: BlockingRecv error: recv got EOF from 127.0.0.1:58844 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 22:49:24.169083 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.162891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.21.27.189:34659)
0809 22:49:24.163195 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.163202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.163577 (+ 375us) server_negotiation.cc:407] Connection header received
0809 22:49:24.163745 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.163751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.163873 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.164014 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.164365 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.165506 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.166859 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.167680 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.167827 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.168256 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.168387 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.168504 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.168622 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.168718 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260809 22:49:24.186419 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.183969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.21.27.188:34685)
0809 22:49:24.184213 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.184223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.184764 (+ 541us) server_negotiation.cc:407] Connection header received
0809 22:49:24.184982 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.184988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.185068 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.185169 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.186152 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58852: BlockingRecv error: recv got EOF from 127.0.0.1:58852 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:49:24.259509 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.254173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50378 (local address 127.21.27.189:34659)
0809 22:49:24.254394 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.254403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.254968 (+ 565us) server_negotiation.cc:407] Connection header received
0809 22:49:24.255233 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.255237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.255315 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.255413 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.255847 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.256595 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.257867 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.258265 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.258389 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.258758 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.258851 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.258941 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.259084 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.259196 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 22:49:24.267931 14059 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:24.265606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.21.27.188:34685)
0809 22:49:24.265796 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.265802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.265822 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:49:24.266045 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.266049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.266118 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.266226 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:24.266930 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58866: BlockingRecv error: recv got EOF from 127.0.0.1:58866 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:49:24.386766 14029 catalog_manager.cc:1259] Loaded cluster ID: 129ee579da704ae9b3f135058207a46d
I20260809 22:49:24.386894 14029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: loading cluster ID for follower catalog manager: success
I20260809 22:49:24.391359 14029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: acquiring CA information for follower catalog manager: success
I20260809 22:49:24.394199 14029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 648f15c5c5ae40fdb13c607302f81ded: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:49:24.406646 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.400163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.21.27.189:34659)
0809 22:49:24.400417 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.400426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.400709 (+ 283us) server_negotiation.cc:407] Connection header received
0809 22:49:24.400886 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.400895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.401006 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.401136 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.401523 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.402864 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.404302 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.405169 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.405328 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.405685 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.405882 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.405984 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.406159 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.406264 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:49:24.420653 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.413166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.21.27.188:34685)
0809 22:49:24.413416 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.413423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.413993 (+ 570us) server_negotiation.cc:407] Connection header received
0809 22:49:24.414104 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.414108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.414331 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.414431 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.415073 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.416090 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.418409 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.418788 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.418876 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.419534 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.420040 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.420148 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.420287 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.420351 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:49:24.438871 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.428697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.21.27.190:40149)
0809 22:49:24.429486 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.429493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.430106 (+ 613us) server_negotiation.cc:407] Connection header received
0809 22:49:24.430364 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.430373 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.430614 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.430829 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.431466 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.433080 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.436252 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.436937 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.437065 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.437730 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.438172 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.438301 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.438457 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.438544 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":541,"threads_started":1}
I20260809 22:49:24.441354 13929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.428858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.21.27.189:34659)
0809 22:49:24.429084 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.429092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.433817 (+ 4725us) server_negotiation.cc:407] Connection header received
0809 22:49:24.434099 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.434106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.434229 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.434298 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.435344 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.436563 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.439106 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.439752 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.439847 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.440498 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.440628 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.440710 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.440957 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.441042 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:49:24.444654 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13771
I20260809 22:49:24.445102 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:24.433969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.21.27.188:34685)
0809 22:49:24.434144 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:24.434149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:24.436423 (+ 2274us) server_negotiation.cc:407] Connection header received
0809 22:49:24.436737 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:24.436743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:24.436830 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:24.436924 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:24.438821 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.440141 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.442671 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:24.443522 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:24.443685 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:24.444144 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:24.444309 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:24.444402 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:24.444641 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 22:49:24.444740 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:49:24.445613 13840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.561275 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13771
I20260809 22:49:24.585631 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13853
I20260809 22:49:24.586766 13919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.695533 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13853
I20260809 22:49:24.718046 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 13934
I20260809 22:49:24.719233 14005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:24.824966 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 13934
2026-08-09T22:49:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4544 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:49:24 dist-test-slave-chsk krb5kdc[14214](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:49:24 dist-test-slave-chsk krb5kdc[14214](info): set up 2 sockets
Aug 09 22:49:24 dist-test-slave-chsk krb5kdc[14214](info): commencing operation
krb5kdc: starting...
W20260809 22:49:25.999504 21614 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/test-user.keytab.
Aug 09 22:49:26 dist-test-slave-chsk krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:49:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.27.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
Entry for principal kudu/127.21.27.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab.
I20260809 22:49:26.338358 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41741
--webserver_interface=127.21.27.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:49:26.677238 14228 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:26.677624 14228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:26.677762 14228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:26.677902 14228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:26.685009 14228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:26.685122 14228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:26.685230 14228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:26.685325 14228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:26.685416 14228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:26.694821 14228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.190.keytab
--principal=kudu/127.21.27.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.190:41741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.27.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:26.696110 14228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:26.697399 14228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:26.708825 14236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.708876 14234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:26.708978 14233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:26.709949 14228 server_base.cc:1034] running on GCE node
I20260809 22:49:26.710754 14228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:26.712141 14228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:26.713369 14228 hybrid_clock.cc:648] HybridClock initialized: now 1786315766713304 us; error 31 us; skew 500 ppm
I20260809 22:49:26.713712 14228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:26 dist-test-slave-chsk krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:26.722219 14228 init.cc:377] Logged in from keytab as kudu/127.21.27.190@KRBTEST.COM (short username kudu)
I20260809 22:49:26.723807 14228 webserver.cc:460] Webserver started at http://127.21.27.190:39057/ using document root <none> and password file <none>
I20260809 22:49:26.724510 14228 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:26.724630 14228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:26.724969 14228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:26.728046 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data/instance:
uuid: "fc56426d9ea64badb020e2e060c0acb2"
format_stamp: "Formatted at 2026-08-09 22:49:26 on dist-test-slave-chsk"
server_key: "f47fc071c4316dba99ab9682926a7b98"
server_key_iv: "cd28ef25512f1065cc697f6099a598d9"
server_key_version: "encryptionkey@0"
I20260809 22:49:26.729028 14228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance:
uuid: "fc56426d9ea64badb020e2e060c0acb2"
format_stamp: "Formatted at 2026-08-09 22:49:26 on dist-test-slave-chsk"
server_key: "f47fc071c4316dba99ab9682926a7b98"
server_key_iv: "cd28ef25512f1065cc697f6099a598d9"
server_key_version: "encryptionkey@0"
I20260809 22:49:26.734172 14228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:49:26.737597 14245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:26.738847 14228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:49:26.739075 14228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal
uuid: "fc56426d9ea64badb020e2e060c0acb2"
format_stamp: "Formatted at 2026-08-09 22:49:26 on dist-test-slave-chsk"
server_key: "f47fc071c4316dba99ab9682926a7b98"
server_key_iv: "cd28ef25512f1065cc697f6099a598d9"
server_key_version: "encryptionkey@0"
I20260809 22:49:26.739343 14228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:26.764086 14228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:26.768198 14228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:26.768469 14228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:26.787245 14228 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.190:41741
I20260809 22:49:26.787357 14303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.190:41741 every 8 connection(s)
I20260809 22:49:26.788604 14228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/data/info.pb
I20260809 22:49:26.791471 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 14228
I20260809 22:49:26.791671 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-0/wal/instance
I20260809 22:49:26.792173 21614 external_mini_cluster.cc:1428] Setting key de55ea5bee1b4790b381bca8b84051b2
I20260809 22:49:26.792363 14304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:26.798684 14304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:26 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:26.823598 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.801829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.21.27.190:41741)
0809 22:49:26.803543 (+ 1714us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:26.803554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:26.803612 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:49:26.805208 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:26.805214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:26.805440 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:26.805605 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:26.807299 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.808525 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.810818 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.811460 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.811601 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.816143 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:26.816151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:26.816166 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:26.816202 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:26.819023 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.820016 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.820022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.820024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.820202 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:26.820668 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:26.820671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:26.820673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:26.821254 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:26.821465 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:26.822179 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 22:49:26.822459 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1479,"spinlock_wait_cycles":87296,"thread_start_us":1289,"threads_started":1}
I20260809 22:49:26.823637 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.801839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41741 (local address 127.0.0.1:43084)
0809 22:49:26.802823 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.802898 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:26.804827 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:26.805682 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:26.805691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:26.806545 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:26.807037 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.807062 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.808693 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.808700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.810675 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:26.810693 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:26.811719 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:26.811733 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:26.811990 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:26.812721 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:26.812756 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:26.815884 (+ 3128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:26.819220 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.819236 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.819251 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.819880 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.820322 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:26.820330 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:26.820337 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:26.820558 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:26.821425 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:26.821437 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:26.821731 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 22:49:26.822051 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 22:49:26.822700 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":56,"thread_start_us":422,"threads_started":1}
WARNING: no policy specified for kudu/127.21.27.189@KRBTEST.COM; defaulting to no policy
I20260809 22:49:26.826596 14304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:26.827713 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.827361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52098)
0809 22:49:26.827508 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.827604 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:49:26.828004 14249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.189@KRBTEST.COM" created.
I20260809 22:49:26.829567 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.829233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52108)
0809 22:49:26.829378 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.829444 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:49:26.829844 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:46611: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
Entry for principal kudu/127.21.27.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab.
I20260809 22:49:26.866254 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:46611
--webserver_interface=127.21.27.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:26.879266 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } attempt: 1
I20260809 22:49:26.880617 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.879955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52120)
0809 22:49:26.880233 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.880427 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 22:49:26.881758 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.881307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52126)
0809 22:49:26.881481 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.881575 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 22:49:26.881981 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:46611: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
I20260809 22:49:26.950297 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } attempt: 2
I20260809 22:49:26.951663 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.951022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52134)
0809 22:49:26.951330 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.951446 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 22:49:26.952819 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:26.952312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52136)
0809 22:49:26.952488 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:26.952619 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 22:49:26.953038 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:46611: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
I20260809 22:49:27.044301 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } attempt: 3
I20260809 22:49:27.045678 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.044989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52142)
0809 22:49:27.045253 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.045453 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 22:49:27.046904 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.046394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52146)
0809 22:49:27.046591 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.046702 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:49:27.047166 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:46611: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
I20260809 22:49:27.190454 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } attempt: 4
I20260809 22:49:27.191919 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.191244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52148)
0809 22:49:27.191625 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.191737 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 22:49:27.192998 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.192600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52158)
0809 22:49:27.192759 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.192852 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 22:49:27.193241 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.189:46611: Network error: Client connection negotiation failed: client connection to 127.21.27.189:46611: connect: Connection refused (error 111)
W20260809 22:49:27.204584 14311 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:27.204943 14311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:27.205065 14311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:27.205204 14311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:49:27.212399 14311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:27.212522 14311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:27.212626 14311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:27.212721 14311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:27.212812 14311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:27.222546 14311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.189.keytab
--principal=kudu/127.21.27.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.189:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.27.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:27.223866 14311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:27.225162 14311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:49:27.236711 14320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.236752 14318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.236792 14316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:27.238013 14311 server_base.cc:1034] running on GCE node
I20260809 22:49:27.238727 14311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:27.240180 14311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:27.241431 14311 hybrid_clock.cc:648] HybridClock initialized: now 1786315767241331 us; error 53 us; skew 500 ppm
I20260809 22:49:27.241740 14311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:27.245040 14311 init.cc:377] Logged in from keytab as kudu/127.21.27.189@KRBTEST.COM (short username kudu)
I20260809 22:49:27.246708 14311 webserver.cc:460] Webserver started at http://127.21.27.189:38235/ using document root <none> and password file <none>
I20260809 22:49:27.247510 14311 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:27.247661 14311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:27.247983 14311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:27.250891 14311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data/instance:
uuid: "7df5e92631da4f22b3f6d0893c5f45fd"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "f97f8763b31d24dcf6805ac85be5616c"
server_key_iv: "5027b1470b6ba7eacf532183154ca1f6"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.251736 14311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal/instance:
uuid: "7df5e92631da4f22b3f6d0893c5f45fd"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "f97f8763b31d24dcf6805ac85be5616c"
server_key_iv: "5027b1470b6ba7eacf532183154ca1f6"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.256739 14311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:49:27.260064 14328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:27.261296 14311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:49:27.261515 14311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal
uuid: "7df5e92631da4f22b3f6d0893c5f45fd"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "f97f8763b31d24dcf6805ac85be5616c"
server_key_iv: "5027b1470b6ba7eacf532183154ca1f6"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.261727 14311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:27.276253 14311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:27.280272 14311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:27.280555 14311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:27.299074 14311 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.189:46611
I20260809 22:49:27.299077 14386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.189:46611 every 8 connection(s)
I20260809 22:49:27.300529 14311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/data/info.pb
I20260809 22:49:27.304006 14387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:27.308892 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 14311
I20260809 22:49:27.309083 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-1/wal/instance
I20260809 22:49:27.309615 21614 external_mini_cluster.cc:1428] Setting key d355ad4999370ef6dcaa70e271cf4b46
I20260809 22:49:27.310892 14387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:27.332298 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.314150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.21.27.190:41741)
0809 22:49:27.314375 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.314382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.315721 (+ 1339us) server_negotiation.cc:407] Connection header received
0809 22:49:27.317578 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.317586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.317685 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.317806 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.319255 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.320364 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.321928 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.322657 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.322796 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.326694 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.326704 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.326707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.326738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.329412 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.330128 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.330133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.330135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.330248 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.330646 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.330650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.330657 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.330926 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.331092 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.331770 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.331987 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 22:49:27.333158 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.314665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41741 (local address 127.0.0.1:43088)
0809 22:49:27.315501 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.315601 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.317321 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.317998 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.318005 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.318540 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.319067 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.319082 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.320550 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.320554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.321775 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.321788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.322871 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.322876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.323074 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.323815 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.323856 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.326481 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.329550 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.329559 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.329573 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.330001 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.330353 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.330359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.330362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.330532 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.331052 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.331060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.331427 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.331664 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.332114 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":375,"threads_started":1}
I20260809 22:49:27.334784 14387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.27.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.27.188@KRBTEST.COM" created.
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:27.350662 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.335603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:52164)
0809 22:49:27.335775 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.335789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.335864 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.337046 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.337053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.337567 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.337783 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.337797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.339698 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.339704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.341194 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.341212 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.341978 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.341990 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.342177 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.342745 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.342769 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.345215 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.348269 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.348275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.348279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.348612 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.348968 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.348973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.348976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.349100 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.349661 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.349666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.349801 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.349943 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.350274 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 22:49:27.350826 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.335917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52164 (local address 127.21.27.189:46611)
0809 22:49:27.336591 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.336596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.336619 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:49:27.336656 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.336661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.336859 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.337013 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.337981 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.339562 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.341335 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.341827 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.341919 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.345408 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.345418 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.345429 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.345455 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.348120 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.348736 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.348745 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.348751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.348866 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.349194 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.349208 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.349211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.349523 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.349728 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.350105 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.350387 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":546,"thread_start_us":391,"threads_started":1}
I20260809 22:49:27.353188 14387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:27.354295 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.353874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:36622)
0809 22:49:27.354023 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.354145 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:49:27.354543 14333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.356040 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.355739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:36630)
0809 22:49:27.355880 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.355951 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:49:27.356240 14387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
Entry for principal kudu/127.21.27.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab.
I20260809 22:49:27.378257 21614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
/tmp/dist-test-taskvPTdCg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:45921
--webserver_interface=127.21.27.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.27.148:37007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:49:27.405521 14387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 1
I20260809 22:49:27.406687 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.406250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54468)
0809 22:49:27.406434 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.406539 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 22:49:27.407655 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.407335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54474)
0809 22:49:27.407467 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.407554 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 22:49:27.407879 14387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.476091 14387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 2
I20260809 22:49:27.477403 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.476715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54476)
0809 22:49:27.476911 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.477016 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 22:49:27.478430 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.478083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54484)
0809 22:49:27.478232 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.478315 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:49:27.478652 14387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.492473 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } attempt: 5
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:27.505342 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.493138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.21.27.189:46611)
0809 22:49:27.493309 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.493317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.493375 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:49:27.493618 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.493622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.493686 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.493787 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.494571 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.495357 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.496467 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.496901 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.497029 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.500240 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.500245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.500248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.500279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.502773 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.503482 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.503493 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.503495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.503623 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.503981 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.503986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.503990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.504273 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.504439 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.504818 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.505031 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 22:49:27.505365 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.493098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:43682)
0809 22:49:27.493294 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.493315 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.493459 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.493819 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.493823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.494238 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.494415 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.494424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.495481 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.495486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.496324 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.496335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.497042 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.497047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.497298 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.497918 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.497935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.500054 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.502910 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.502915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.502919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.503364 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.503732 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.503737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.503741 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.503875 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.504393 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.504398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.504555 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.504705 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.505010 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 22:49:27.506635 14304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:27.507602 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.507258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54496)
0809 22:49:27.507404 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.507478 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:49:27.508443 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.508132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54504)
0809 22:49:27.508281 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.508341 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 22:49:27.508664 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.559896 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 1
I20260809 22:49:27.561318 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.560632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54512)
0809 22:49:27.560902 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.561056 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 22:49:27.562533 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.562140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54528)
0809 22:49:27.562310 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.562397 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 22:49:27.562780 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.569885 14387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 3
I20260809 22:49:27.570986 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.570538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54538)
0809 22:49:27.570742 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.570836 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 22:49:27.572220 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.571787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54550)
0809 22:49:27.572015 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.572105 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 22:49:27.572438 14387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
I20260809 22:49:27.631049 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 2
I20260809 22:49:27.632419 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.631846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54566)
0809 22:49:27.632145 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.632249 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 22:49:27.633549 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.633206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54582)
0809 22:49:27.633361 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.633431 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 22:49:27.633816 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
W20260809 22:49:27.713284 14394 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 22:49:27.713598 14394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:49:27.713670 14394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:49:27.713752 14394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 22:49:27.715659 14387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 4
I20260809 22:49:27.716765 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.716292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54584)
0809 22:49:27.716526 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.716617 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:49:27.717813 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.717473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54592)
0809 22:49:27.717637 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.717709 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 22:49:27.718050 14387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
W20260809 22:49:27.721304 14394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:49:27.721417 14394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:49:27.721532 14394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:49:27.721613 14394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:49:27.721715 14394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:49:27.734263 14394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.27.148:37007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.27.190:41741,127.21.27.189:46611,127.21.27.188:45921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/krb5kdc/kudu_127.21.27.188.keytab
--principal=kudu/127.21.27.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.27.188:45921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.27.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:48:06 UTC on ccycloud.quasar-tdsnhu.root.comops.site
build id 4576819
ASAN enabled
I20260809 22:49:27.735605 14394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:49:27.736874 14394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 22:49:27.740067 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 3
I20260809 22:49:27.741240 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.740739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54608)
0809 22:49:27.740924 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.741049 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:49:27.742610 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.742030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54624)
0809 22:49:27.742341 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.742446 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260809 22:49:27.742998 14304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.27.188:45921: Network error: Client connection negotiation failed: client connection to 127.21.27.188:45921: connect: Connection refused (error 111)
W20260809 22:49:27.749207 14402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.749208 14405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:49:27.749402 14403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:49:27.750034 14394 server_base.cc:1034] running on GCE node
I20260809 22:49:27.750788 14394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:49:27.752259 14394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:49:27.753500 14394 hybrid_clock.cc:648] HybridClock initialized: now 1786315767753435 us; error 42 us; skew 500 ppm
I20260809 22:49:27.753835 14394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:49:27.757018 14394 init.cc:377] Logged in from keytab as kudu/127.21.27.188@KRBTEST.COM (short username kudu)
I20260809 22:49:27.758474 14394 webserver.cc:460] Webserver started at http://127.21.27.188:33943/ using document root <none> and password file <none>
I20260809 22:49:27.759236 14394 fs_manager.cc:359] Metadata directory not provided
I20260809 22:49:27.759368 14394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:49:27.759697 14394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:49:27.762619 14394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data/instance:
uuid: "1869d3711b95491f8a437339df8a79d3"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "c8aa30b045432937f7831f8c4a7e751e"
server_key_iv: "5cc54e33b995ad87f41d27c3244b79fb"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.763460 14394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal/instance:
uuid: "1869d3711b95491f8a437339df8a79d3"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "c8aa30b045432937f7831f8c4a7e751e"
server_key_iv: "5cc54e33b995ad87f41d27c3244b79fb"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.768684 14394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:49:27.772133 14412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:49:27.773371 14394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:49:27.773551 14394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data,/tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal
uuid: "1869d3711b95491f8a437339df8a79d3"
format_stamp: "Formatted at 2026-08-09 22:49:27 on dist-test-slave-chsk"
server_key: "c8aa30b045432937f7831f8c4a7e751e"
server_key_iv: "5cc54e33b995ad87f41d27c3244b79fb"
server_key_version: "encryptionkey@0"
I20260809 22:49:27.773772 14394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:49:27.802383 14394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:49:27.806874 14394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:49:27.807109 14394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:49:27.826036 14394 rpc_server.cc:307] RPC server started. Bound to: 127.21.27.188:45921
I20260809 22:49:27.826061 14468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.27.188:45921 every 8 connection(s)
I20260809 22:49:27.827414 14394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/data/info.pb
I20260809 22:49:27.831032 14469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:49:27.835317 21614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu as pid 14394
I20260809 22:49:27.835507 21614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvPTdCg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786315394803050-21614-0/minicluster-data/master-2/wal/instance
I20260809 22:49:27.836066 21614 external_mini_cluster.cc:1428] Setting key e2801a9a6f69031ddda935a660545f34
I20260809 22:49:27.838739 14469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:49:27.858340 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.842686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.21.27.190:41741)
0809 22:49:27.843325 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.843335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.843712 (+ 377us) server_negotiation.cc:407] Connection header received
0809 22:49:27.843924 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.843930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.843998 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.844107 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.845264 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.845986 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.847563 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.848125 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.848235 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.851971 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.851980 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.851988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.852036 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.855025 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.855825 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.855832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.855835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.856007 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.856440 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.856445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.856448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.856777 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.856905 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.857661 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.857901 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260809 22:49:27.859396 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.842677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.190:41741 (local address 127.0.0.1:51096)
0809 22:49:27.843583 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.843624 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.843801 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.844197 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.844205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.844823 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.845109 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.845124 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.846137 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.846142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.847420 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.847434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.848351 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.848359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.848556 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.849125 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.849159 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.851767 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.855174 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.855185 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.855196 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.855689 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.856113 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.856117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.856119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.856323 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.856901 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.856908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.857227 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.857494 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.858000 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":4,"spinlock_wait_cycles":17280,"thread_start_us":548,"threads_started":1}
I20260809 22:49:27.859416 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.837912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.21.27.188:45921)
0809 22:49:27.838646 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.838657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.838697 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:49:27.840722 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.840745 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.841177 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.841561 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.843443 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.844990 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.846422 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.847059 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.847261 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.852685 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.852692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.852709 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.852735 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.855319 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.856001 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.856006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.856009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.856195 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.856597 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.856600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.856602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.857167 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.857491 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.857752 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.858097 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":535,"thread_start_us":342,"threads_started":1}
I20260809 22:49:27.861094 14469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.190@KRBTEST.COM
I20260809 22:49:27.875109 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.861921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43696 (local address 127.21.27.189:46611)
0809 22:49:27.862127 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.862133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.862198 (+ 65us) server_negotiation.cc:407] Connection header received
0809 22:49:27.862383 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.862389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.862473 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.862573 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.863292 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.864022 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.865369 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.865818 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.865926 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.869350 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.869355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.869359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.869389 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.872582 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.873165 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.873172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.873176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.873351 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.873790 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.873793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.873795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.874043 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.874149 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.874626 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.874800 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:49:27.875316 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.861940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.189:46611 (local address 127.0.0.1:43696)
0809 22:49:27.862119 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.862139 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.862236 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.862629 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.862634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.862979 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.863112 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.863120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.864156 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.864164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.865169 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.865182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.865971 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.865977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.866208 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.866793 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.866809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.869169 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.872708 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.872712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.872714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.873051 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.873475 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.873481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.873484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.873682 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.874158 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.874163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.874379 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.874541 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.874824 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:49:27.876477 14469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } has no permanent_uuid. Determining permanent_uuid...
I20260809 22:49:27.877931 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.862459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.21.27.190:41741)
0809 22:49:27.862618 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.862625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.862647 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:49:27.862835 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.862840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.862913 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.863007 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.864176 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.864835 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.866355 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.866986 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.867093 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.871557 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.871564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.871568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.871620 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.874911 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.875790 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.875795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.875797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.875937 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.876389 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.876393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.876396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.876790 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.877064 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.877424 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.877628 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.188@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.27.189@KRBTEST.COM
I20260809 22:49:27.890983 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.877280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.21.27.188:45921)
0809 22:49:27.877465 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.877471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.877831 (+ 360us) server_negotiation.cc:407] Connection header received
0809 22:49:27.877995 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.878004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.878076 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.878156 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.878923 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.879958 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.882073 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.882489 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.882616 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.885671 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.885675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.885678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.885706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.888392 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.889229 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.889232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.889234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.889355 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.889780 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.889786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.889791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.890054 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.890160 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.890602 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.890758 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":74}
I20260809 22:49:27.891422 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.877390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54640)
0809 22:49:27.877738 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.877757 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.877882 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.878277 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.878280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.878662 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.878802 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.878810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.881111 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.881115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.881924 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.881934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.882623 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.882628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.882869 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.883394 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.883410 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.885506 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.888554 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.888562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.888568 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.889104 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.889465 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.889471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.889476 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.889649 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.890174 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.890179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.890346 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.890514 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.890956 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":117}
I20260809 22:49:27.894220 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.879702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.21.27.189:46611)
0809 22:49:27.879853 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.879857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.879871 (+ 14us) server_negotiation.cc:407] Connection header received
0809 22:49:27.879908 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.879911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.879959 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.880047 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.881080 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.881764 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.882934 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.883529 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.883668 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.887923 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.887928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.887931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.887975 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.891481 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.892305 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.892310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.892313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.892482 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.892881 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.892886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.892890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.893141 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.893426 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.893706 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.893934 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:49:27.896904 14469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: Bootstrap starting.
I20260809 22:49:27.898864 21614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:49:27.900401 14469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:27.901587 14469 log.cc:824] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:27.901273 14276 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fc56426d9ea64badb020e2e060c0acb2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51106
I20260809 22:49:27.904345 14469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: No bootstrap required, opened a new log
I20260809 22:49:27.914865 14469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.915336 14469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1869d3711b95491f8a437339df8a79d3, State: Initialized, Role: FOLLOWER
I20260809 22:49:27.915804 14469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.916725 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:27.917043 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:27.917970 14469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:27.920256 14304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 4
W20260809 22:49:27.929113 14490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:49:27.929379 14490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 22:49:27 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.190@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:49:27.926709 14394 master_runner.cc:186] Error getting master registration for 127.21.27.190:41741: OK, instance_id { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" instance_seqno: 1786315766781965 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:27.936522 14394 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:27.940472 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.921055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54646)
0809 22:49:27.921261 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:27.921283 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:27.921439 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:27.921864 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:27.921869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:27.922500 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:27.922754 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.922767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.924325 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.924332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.925806 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:27.925822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.927195 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.927201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:27.927393 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.928097 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:27.928122 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:27.931102 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:27.937479 (+ 6377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.937485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.937489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.938052 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.938478 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:27.938483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:27.938486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:27.938675 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:27.939400 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:27.939404 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:27.939598 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 22:49:27.939751 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 22:49:27.940059 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:49:27.940479 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:27.921134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.21.27.188:45921)
0809 22:49:27.921299 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:27.921307 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:27.921384 (+ 77us) server_negotiation.cc:407] Connection header received
0809 22:49:27.921634 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:27.921640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:27.921721 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:27.921818 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:27.922908 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.924150 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.925946 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:27.926848 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:27.926984 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:27.931341 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:27.931348 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:27.931353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:27.931394 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:27.937319 (+ 5925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.938181 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.938187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.938191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.938358 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:27.938788 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:27.938794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:27.938798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:27.939285 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:27.939441 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:27.939867 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 22:49:27.940104 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:49:27.947021 14304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: Bootstrap starting.
I20260809 22:49:27.951354 14304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:27.952690 14304 log.cc:824] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:27.955734 14304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: No bootstrap required, opened a new log
I20260809 22:49:27.960853 14304 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.961212 14304 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc56426d9ea64badb020e2e060c0acb2, State: Initialized, Role: FOLLOWER
I20260809 22:49:27.961724 14304 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.962749 14493 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:27.963311 14493 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:27.963891 14304 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:27.964000 14276 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 22:49:27.964179 14276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:27.968721 14276 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.970629 14276 leader_election.cc:288] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [CANDIDATE]: Term 1 election: Requested vote from peers 7df5e92631da4f22b3f6d0893c5f45fd (127.21.27.189:46611), 1869d3711b95491f8a437339df8a79d3 (127.21.27.188:45921)
I20260809 22:49:27.972539 14441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc56426d9ea64badb020e2e060c0acb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1869d3711b95491f8a437339df8a79d3"
I20260809 22:49:27.973067 14441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:27.976831 14441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc56426d9ea64badb020e2e060c0acb2 in term 1.
I20260809 22:49:27.977783 14250 leader_election.cc:302] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1869d3711b95491f8a437339df8a79d3, fc56426d9ea64badb020e2e060c0acb2; no voters:
I20260809 22:49:27.978278 14493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:49:27.988482 14493 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [term 1 LEADER]: Becoming Leader. State: Replica: fc56426d9ea64badb020e2e060c0acb2, State: Running, Role: LEADER
I20260809 22:49:27.989166 14493 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:27.997006 14501 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc56426d9ea64badb020e2e060c0acb2. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:27.997478 14501 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:28.003690 14510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:49:28.005934 14512 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:49:28.017246 14387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } attempt: 5
W20260809 22:49:28.021283 14252 tablet.cc:2369] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:28.010536 14228 master_runner.cc:186] Error getting master registration for 127.21.27.189:46611: OK, instance_id { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" instance_seqno: 1786315767293889 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 22:49:28.031947 14228 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 22:49:28.040146 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:28.008478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.21.27.190:41741)
0809 22:49:28.008690 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.008697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.011817 (+ 3120us) server_negotiation.cc:407] Connection header received
0809 22:49:28.012345 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.012353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.012463 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.012551 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.017330 (+ 4779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.018740 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.022935 (+ 4195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.023676 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.023790 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:28.030015 (+ 6225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:28.030021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:28.030026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:28.030068 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:28.034143 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.035632 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.035638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.035666 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.035973 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.036719 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.036752 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.036758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.037431 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:28.037918 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:28.038953 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0809 22:49:28.039401 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 22:49:28.040614 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:28.010424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43710 (local address 127.21.27.189:46611)
0809 22:49:28.010611 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.010618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.021990 (+ 11372us) server_negotiation.cc:407] Connection header received
0809 22:49:28.022808 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.022815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.022908 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.022976 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.025322 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.026481 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.028922 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.029640 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.029740 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:28.033345 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:28.033352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:28.033356 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:28.033397 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:28.037204 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.038077 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.038082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.038085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.038272 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.038731 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.038737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.038743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.039094 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:28.039351 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:28.039903 (+ 552us) server_negotiation.cc:299] Negotiation successful
0809 22:49:28.040153 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 22:49:28 dist-test-slave-chsk krb5kdc[14214](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786315767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.27.189@KRBTEST.COM for kudu/127.21.27.188@KRBTEST.COM
I20260809 22:49:28.050904 14510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:49:28.054147 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:28.013800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.21.27.188:45921)
0809 22:49:28.013997 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.014004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.022826 (+ 8822us) server_negotiation.cc:407] Connection header received
0809 22:49:28.023100 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.023109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.023248 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.023345 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.025716 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.026759 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.029465 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.030348 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.030465 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:28.043547 (+ 13082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:28.043553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:28.043558 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:28.043606 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:28.047609 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.048477 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.048490 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.048531 (+ 41us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.048916 (+ 385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.049618 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.049671 (+ 53us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.049676 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.050617 (+ 941us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:28.052691 (+ 2074us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:28.052936 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 22:49:28.053348 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:49:28.055923 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:28.017853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.27.188:45921 (local address 127.0.0.1:54662)
0809 22:49:28.018189 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 22:49:28.018209 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:49:28.018313 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:49:28.037352 (+ 19039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:49:28.037359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:49:28.037856 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:49:28.038056 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:28.038068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.040543 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.040549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:28.041986 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:49:28.042017 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.043349 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.043357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:49:28.043593 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:28.044314 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:49:28.044342 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:49:28.048374 (+ 4032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:49:28.053052 (+ 4678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:28.053059 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:28.053063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:28.053604 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:28.054090 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:49:28.054096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:49:28.054101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:49:28.054274 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:49:28.054869 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:49:28.054874 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:49:28.055024 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 22:49:28.055254 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 22:49:28.055546 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260809 22:49:28.056324 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:28.018515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.21.27.188:45921)
0809 22:49:28.036895 (+ 18380us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.036901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.036925 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:49:28.036972 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.036977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.037052 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.038207 (+ 1155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.038463 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.039743 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.042412 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:28.043180 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:28.043306 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:28.048614 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:49:28.048620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:49:28.048627 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:49:28.048667 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:49:28.052904 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.053766 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.053771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.053774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.053949 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:49:28.054424 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:49:28.054430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:49:28.054433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:49:28.054748 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:49:28.054901 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:28.055345 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 22:49:28.055551 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18221,"thread_start_us":18640,"threads_started":1}
W20260809 22:49:28.059124 14418 tablet.cc:2369] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:49:28.061867 14387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: Bootstrap starting.
I20260809 22:49:28.065809 14441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [term 1 FOLLOWER]: Refusing update from remote peer fc56426d9ea64badb020e2e060c0acb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:28.065856 14387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: Neither blocks nor log segments found. Creating new log.
I20260809 22:49:28.066838 14493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:28.067114 14387 log.cc:824] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: Log is configured to *not* fsync() on all Append() calls
I20260809 22:49:28.088842 14387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: No bootstrap required, opened a new log
I20260809 22:49:28.101522 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc56426d9ea64badb020e2e060c0acb2. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.101852 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:28.104540 14519 mvcc.cc:205] Tried to move back new op lower bound from 7316749385980260352 to 7316749385689870336. Current Snapshot: MvccSnapshot[applied={T|T < 7316749385980260352}]
I20260809 22:49:28.114679 14387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:28.115473 14387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df5e92631da4f22b3f6d0893c5f45fd, State: Initialized, Role: FOLLOWER
I20260809 22:49:28.116451 14387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } }
I20260809 22:49:28.116798 14501 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.117050 14501 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:28.119489 14510 catalog_manager.cc:1347] Generated new cluster ID: 009359f8d7ee436a92136c162c231b4f
I20260809 22:49:28.119602 14510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:49:28.143226 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.143651 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:28.153400 14387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:49:28.153828 14359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:49:28.160636 14360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fc56426d9ea64badb020e2e060c0acb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7df5e92631da4f22b3f6d0893c5f45fd"
I20260809 22:49:28.158627 14359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [term 1 FOLLOWER]: Refusing update from remote peer fc56426d9ea64badb020e2e060c0acb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:49:28.163997 14493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:49:28.152958 14525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.165319 14525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:28.178308 14360 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc56426d9ea64badb020e2e060c0acb2 in term 1.
I20260809 22:49:28.180593 14510 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:49:28.182324 14510 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:49:28.205117 14510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2: Generated new TSK 0
I20260809 22:49:28.206107 14510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:49:28.243486 14525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc56426d9ea64badb020e2e060c0acb2. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.243878 14525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:28.244095 14493 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.244568 14493 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc56426d9ea64badb020e2e060c0acb2 [sys.catalog]: This master's current role is: LEADER
I20260809 22:49:28.250795 14537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc56426d9ea64badb020e2e060c0acb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc56426d9ea64badb020e2e060c0acb2" member_type: VOTER last_known_addr { host: "127.21.27.190" port: 41741 } } peers { permanent_uuid: "7df5e92631da4f22b3f6d0893c5f45fd" member_type: VOTER last_known_addr { host: "127.21.27.189" port: 46611 } } peers { permanent_uuid: "1869d3711b95491f8a437339df8a79d3" member_type: VOTER last_known_addr { host: "127.21.27.188" port: 45921 } } }
I20260809 22:49:28.252249 14537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:49:28.273499 14542 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 22:49:28.333911 21614 mini_kdc.cc:348] Time spent kdestroy: real 0.106s user 0.000s sys 0.032s
W20260809 22:49:28.351276 14473 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.338185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51136 (local address 127.21.27.190:41741)
0809 22:49:28.338430 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.338436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.349099 (+ 10663us) server_negotiation.cc:407] Connection header received
0809 22:49:28.349599 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.349606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.349702 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.349784 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.350996 (+ 1212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51136: BlockingRecv error: recv got EOF from 127.0.0.1:51136 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 22:49:28.354395 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.337977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43718 (local address 127.21.27.189:46611)
0809 22:49:28.338204 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.338211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.352147 (+ 13936us) server_negotiation.cc:407] Connection header received
0809 22:49:28.352795 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.352801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.352889 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.352958 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.354167 (+ 1209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43718: BlockingRecv error: recv got EOF from 127.0.0.1:43718 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 22:49:28.354657 14518 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.344643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.21.27.188:45921)
0809 22:49:28.344864 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.344872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.352453 (+ 7581us) server_negotiation.cc:407] Connection header received
0809 22:49:28.352920 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.352927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.353037 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.353119 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.354378 (+ 1259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54674: BlockingRecv error: recv got EOF from 127.0.0.1:54674 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 22:49:28.361918 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.359673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.21.27.189:46611)
0809 22:49:28.359885 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.359891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.360446 (+ 555us) server_negotiation.cc:407] Connection header received
0809 22:49:28.360662 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.360668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.360788 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.360895 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.361660 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43732: BlockingRecv error: recv got EOF from 127.0.0.1:43732 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:49:28.368510 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.366553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43748 (local address 127.21.27.189:46611)
0809 22:49:28.366723 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.366729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.367043 (+ 314us) server_negotiation.cc:407] Connection header received
0809 22:49:28.367259 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.367264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.367356 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.367477 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.368303 (+ 826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43748: BlockingRecv error: recv got EOF from 127.0.0.1:43748 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 22:49:28.379483 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.377142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43754 (local address 127.21.27.189:46611)
0809 22:49:28.377327 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.377335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.377871 (+ 536us) server_negotiation.cc:407] Connection header received
0809 22:49:28.378092 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.378098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.378221 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.378358 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.379216 (+ 858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43754: BlockingRecv error: recv got EOF from 127.0.0.1:43754 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 22:49:28.391273 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.388941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.21.27.189:46611)
0809 22:49:28.389187 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.389194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.389726 (+ 532us) server_negotiation.cc:407] Connection header received
0809 22:49:28.389930 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.389936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.390038 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.390157 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.391013 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43764: BlockingRecv error: recv got EOF from 127.0.0.1:43764 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260809 22:49:28.407313 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.404903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.21.27.189:46611)
0809 22:49:28.405088 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.405094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.405638 (+ 544us) server_negotiation.cc:407] Connection header received
0809 22:49:28.405836 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.405842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.405942 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.406066 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.407019 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43776: BlockingRecv error: recv got EOF from 127.0.0.1:43776 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 22:49:28.435115 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.432777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.21.27.189:46611)
0809 22:49:28.432985 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.432993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.433536 (+ 543us) server_negotiation.cc:407] Connection header received
0809 22:49:28.433768 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.433775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.433876 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.433991 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.434858 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43782: BlockingRecv error: recv got EOF from 127.0.0.1:43782 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 22:49:28.477216 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.474660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.21.27.189:46611)
0809 22:49:28.474865 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.474872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.475482 (+ 610us) server_negotiation.cc:407] Connection header received
0809 22:49:28.475712 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.475718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.475825 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.475943 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.476954 (+ 1011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43784: BlockingRecv error: recv got EOF from 127.0.0.1:43784 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:49:28.547449 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.545261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.21.27.189:46611)
0809 22:49:28.545507 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.545513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.545871 (+ 358us) server_negotiation.cc:407] Connection header received
0809 22:49:28.546112 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.546116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.546190 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.546280 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.547224 (+ 944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43792: BlockingRecv error: recv got EOF from 127.0.0.1:43792 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 22:49:28.681233 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.679187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.21.27.189:46611)
0809 22:49:28.679439 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.679447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.679796 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:49:28.680033 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.680039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.680123 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.680234 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.681012 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43798: BlockingRecv error: recv got EOF from 127.0.0.1:43798 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:49:28.931633 14490 catalog_manager.cc:1259] Loaded cluster ID: 009359f8d7ee436a92136c162c231b4f
I20260809 22:49:28.931746 14490 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: loading cluster ID for follower catalog manager: success
I20260809 22:49:28.934199 14490 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: acquiring CA information for follower catalog manager: success
I20260809 22:49:28.935918 14490 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1869d3711b95491f8a437339df8a79d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 22:49:28.945147 14391 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:49:28.943419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.21.27.189:46611)
0809 22:49:28.943659 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:28.943666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:28.943854 (+ 188us) server_negotiation.cc:407] Connection header received
0809 22:49:28.944020 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:28.944024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:28.944102 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:28.944190 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:49:28.944897 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43806: BlockingRecv error: recv got EOF from 127.0.0.1:43806 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 22:49:29.277959 14542 catalog_manager.cc:1259] Loaded cluster ID: 009359f8d7ee436a92136c162c231b4f
I20260809 22:49:29.278127 14542 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: loading cluster ID for follower catalog manager: success
I20260809 22:49:29.280989 14542 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: acquiring CA information for follower catalog manager: success
I20260809 22:49:29.282771 14542 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7df5e92631da4f22b3f6d0893c5f45fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:49:29.473218 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.465495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.21.27.189:46611)
0809 22:49:29.466473 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:29.466478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:29.466527 (+ 49us) server_negotiation.cc:407] Connection header received
0809 22:49:29.466592 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:29.466597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:29.466729 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:29.466835 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:29.467536 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.468463 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.470567 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.471019 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.471121 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:29.471872 (+ 751us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:29.472459 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:29.472588 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:29.472745 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 22:49:29.472826 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":714,"threads_started":1}
I20260809 22:49:29.485853 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.480137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.21.27.188:45921)
0809 22:49:29.480819 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:29.480826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:29.480846 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:49:29.480951 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:29.480955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:29.481093 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:29.481259 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:29.481657 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.482367 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.483694 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.484096 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.484227 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:29.484731 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:29.485247 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:29.485398 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:29.485478 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 22:49:29.485562 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":432,"threads_started":1}
I20260809 22:49:29.514629 14629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.501536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51138 (local address 127.21.27.190:41741)
0809 22:49:29.502817 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:29.502825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:29.506656 (+ 3831us) server_negotiation.cc:407] Connection header received
0809 22:49:29.506796 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:29.506801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:29.507016 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:29.507275 (+ 259us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:29.508176 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.509312 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.511595 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.512260 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.512416 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:29.513166 (+ 750us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:29.513575 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:29.513677 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:29.514204 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 22:49:29.514298 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1132,"thread_start_us":1020,"threads_started":1}
I20260809 22:49:29.515298 14615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.501852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.21.27.189:46611)
0809 22:49:29.502056 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:29.502065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:29.506203 (+ 4138us) server_negotiation.cc:407] Connection header received
0809 22:49:29.506775 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:29.506783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:29.506904 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:29.506988 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:29.508001 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.509538 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.512833 (+ 3295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.513462 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.513561 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:29.514443 (+ 882us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:29.514566 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:29.514661 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:29.514873 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:49:29.514954 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 22:49:29.520036 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 14228
I20260809 22:49:29.521628 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:49:29.501359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.21.27.188:45921)
0809 22:49:29.501539 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:49:29.501545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:49:29.512945 (+ 11400us) server_negotiation.cc:407] Connection header received
0809 22:49:29.513374 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:49:29.513380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:49:29.513474 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:49:29.513535 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:49:29.514333 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.515421 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.518496 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:49:29.519093 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:49:29.519273 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:49:29.520612 (+ 1339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:49:29.520843 (+ 231us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:49:29.520968 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:49:29.521155 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 22:49:29.521266 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:49:29.522012 14297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:29.673990 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 14228
I20260809 22:49:29.698685 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 14311
I20260809 22:49:29.699854 14382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:29.808609 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 14311
I20260809 22:49:29.831795 21614 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu pid 14394
I20260809 22:49:29.832953 14464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:49:29.939248 21614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvPTdCg/build/asan/bin/kudu with pid 14394
2026-08-09T22:49:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5113 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9657 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (375104 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST